2024-09-16T11:00:20.064 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T11:00:20.082 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T11:00:20.338 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_copy_on_read} email: sangadi@redhat.com first_in_suite: false job_id: '7907154' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: rbd cache: true rbd clone copy on read: true global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - mons down - mon down - out of quorum sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_sangadi@teuthology priority: 130 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9764 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_10:44:09 tube: smithi user: sangadi verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T11:00:20.338 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-16T11:00:20.339 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-16T11:00:20.339 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T11:00:20.341 INFO:teuthology.task.internal:Checking packages... 2024-09-16T11:00:20.366 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-16T11:00:20.366 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T11:00:20.366 INFO:teuthology.packaging:ref: None 2024-09-16T11:00:20.366 INFO:teuthology.packaging:tag: None 2024-09-16T11:00:20.366 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T11:00:20.366 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:00:20.367 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T11:00:20.510 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643-gd75f2382-1focal 2024-09-16T11:00:20.512 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T11:00:20.521 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T11:00:20.521 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T11:00:20.527 INFO:teuthology.task.internal:Saving configuration 2024-09-16T11:00:20.543 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T11:00:20.549 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T11:00:20.569 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 10:54:07.005440', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-16T11:00:20.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 10:54:07.004315', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-16T11:00:20.589 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T11:00:20.595 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-16T11:00:20.595 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-16T11:00:20.596 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T11:00:20.646 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcc87a377f0>, signals=[15]) 2024-09-16T11:00:20.646 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T11:00:20.652 INFO:teuthology.task.internal:Opening connections... 2024-09-16T11:00:20.652 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:00:20.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:00:20.743 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:00:20.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:00:20.830 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T11:00:20.837 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-09-16T11:00:20.844 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-09-16T11:00:20.844 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Ubuntu" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:ID=ubuntu 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE=debian 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="20.04" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T11:00:20.898 INFO:teuthology.orchestra.run.smithi162.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T11:00:20.899 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_CODENAME=focal 2024-09-16T11:00:20.899 INFO:teuthology.orchestra.run.smithi162.stdout:UBUNTU_CODENAME=focal 2024-09-16T11:00:20.899 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-09-16T11:00:20.927 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-09-16T11:00:20.934 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-09-16T11:00:20.935 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-09-16T11:00:20.983 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="Ubuntu" 2024-09-16T11:00:20.983 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-16T11:00:20.983 INFO:teuthology.orchestra.run.smithi165.stdout:ID=ubuntu 2024-09-16T11:00:20.983 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE=debian 2024-09-16T11:00:20.983 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-16T11:00:20.983 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="20.04" 2024-09-16T11:00:20.984 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T11:00:20.984 INFO:teuthology.orchestra.run.smithi165.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T11:00:20.984 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T11:00:20.984 INFO:teuthology.orchestra.run.smithi165.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T11:00:20.985 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_CODENAME=focal 2024-09-16T11:00:20.985 INFO:teuthology.orchestra.run.smithi165.stdout:UBUNTU_CODENAME=focal 2024-09-16T11:00:20.986 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-09-16T11:00:21.010 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T11:00:21.020 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T11:00:21.063 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T11:00:21.063 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-09-16T11:00:21.065 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-09-16T11:00:21.071 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T11:00:21.077 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T11:00:21.077 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-09-16T11:00:21.079 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-09-16T11:00:21.098 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T11:00:21.137 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T11:00:21.152 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T11:00:21.152 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T11:00:21.152 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T11:00:21.153 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T11:00:21.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:00:21.158 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-16T11:00:21.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:00:21.159 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-09-16T11:00:21.165 INFO:teuthology.orchestra.run.smithi162.stdout:5.4.0-124-generic 2024-09-16T11:00:21.165 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.4.0-124-generic 2024-09-16T11:00:21.165 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get clean 2024-09-16T11:00:21.166 INFO:teuthology.orchestra.run.smithi165.stdout:5.4.0-124-generic 2024-09-16T11:00:21.167 INFO:teuthology.task.kernel:Running kernel on smithi165: 5.4.0-124-generic 2024-09-16T11:00:21.167 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get clean 2024-09-16T11:00:21.265 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get update 2024-09-16T11:00:21.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get update 2024-09-16T11:00:21.433 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T11:00:21.436 INFO:teuthology.orchestra.run.smithi165.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-16T11:00:21.436 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-16T11:00:21.507 INFO:teuthology.orchestra.run.smithi165.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-16T11:00:21.577 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T11:00:21.577 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-16T11:00:21.626 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-16T11:00:21.833 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-16T11:00:21.917 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-16T11:00:21.959 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-16T11:00:21.962 INFO:teuthology.orchestra.run.smithi165.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-16T11:00:21.962 INFO:teuthology.orchestra.run.smithi165.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-16T11:00:21.962 INFO:teuthology.orchestra.run.smithi165.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-16T11:00:21.969 INFO:teuthology.orchestra.run.smithi165.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-16T11:00:21.980 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-16T11:00:21.985 INFO:teuthology.orchestra.run.smithi165.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-16T11:00:21.989 INFO:teuthology.orchestra.run.smithi165.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T11:00:21.989 INFO:teuthology.orchestra.run.smithi165.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-16T11:00:21.993 INFO:teuthology.orchestra.run.smithi165.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-16T11:00:22.000 INFO:teuthology.orchestra.run.smithi165.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-16T11:00:22.003 INFO:teuthology.orchestra.run.smithi165.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi165.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-16T11:00:22.004 INFO:teuthology.orchestra.run.smithi165.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-16T11:00:22.005 INFO:teuthology.orchestra.run.smithi165.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-16T11:00:22.011 INFO:teuthology.orchestra.run.smithi165.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-16T11:00:22.055 INFO:teuthology.orchestra.run.smithi165.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-16T11:00:22.059 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-16T11:00:22.094 INFO:teuthology.orchestra.run.smithi165.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-16T11:00:22.097 INFO:teuthology.orchestra.run.smithi165.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-16T11:00:22.097 INFO:teuthology.orchestra.run.smithi165.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-16T11:00:22.118 INFO:teuthology.orchestra.run.smithi165.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-16T11:00:22.118 INFO:teuthology.orchestra.run.smithi165.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-16T11:00:22.121 INFO:teuthology.orchestra.run.smithi165.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T11:00:22.121 INFO:teuthology.orchestra.run.smithi165.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-16T11:00:22.126 INFO:teuthology.orchestra.run.smithi165.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-16T11:00:22.134 INFO:teuthology.orchestra.run.smithi165.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-16T11:00:22.136 INFO:teuthology.orchestra.run.smithi165.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-16T11:00:22.136 INFO:teuthology.orchestra.run.smithi165.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-16T11:00:22.137 INFO:teuthology.orchestra.run.smithi165.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-16T11:00:22.145 INFO:teuthology.orchestra.run.smithi165.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-16T11:00:22.146 INFO:teuthology.orchestra.run.smithi165.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-16T11:00:22.146 INFO:teuthology.orchestra.run.smithi165.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-16T11:00:22.147 INFO:teuthology.orchestra.run.smithi165.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-16T11:00:22.147 INFO:teuthology.orchestra.run.smithi165.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-16T11:00:22.162 INFO:teuthology.orchestra.run.smithi165.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-16T11:00:22.163 INFO:teuthology.orchestra.run.smithi165.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-16T11:00:22.163 INFO:teuthology.orchestra.run.smithi165.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-16T11:00:22.198 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-16T11:00:22.545 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-16T11:00:22.546 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-16T11:00:22.559 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-16T11:00:22.568 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-16T11:00:22.569 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-16T11:00:22.569 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-16T11:00:22.657 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-16T11:00:22.665 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T11:00:22.665 INFO:teuthology.orchestra.run.smithi162.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-16T11:00:22.717 INFO:teuthology.orchestra.run.smithi162.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-16T11:00:22.724 INFO:teuthology.orchestra.run.smithi162.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-16T11:00:22.727 INFO:teuthology.orchestra.run.smithi162.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi162.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi162.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-16T11:00:22.732 INFO:teuthology.orchestra.run.smithi162.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-16T11:00:22.733 INFO:teuthology.orchestra.run.smithi162.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi162.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-16T11:00:22.738 INFO:teuthology.orchestra.run.smithi162.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-16T11:00:22.739 INFO:teuthology.orchestra.run.smithi162.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-16T11:00:22.805 INFO:teuthology.orchestra.run.smithi162.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-16T11:00:22.832 INFO:teuthology.orchestra.run.smithi162.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-16T11:00:22.846 INFO:teuthology.orchestra.run.smithi162.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-16T11:00:22.846 INFO:teuthology.orchestra.run.smithi162.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-16T11:00:22.875 INFO:teuthology.orchestra.run.smithi162.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-16T11:00:22.907 INFO:teuthology.orchestra.run.smithi162.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-16T11:00:22.913 INFO:teuthology.orchestra.run.smithi162.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-16T11:00:22.913 INFO:teuthology.orchestra.run.smithi162.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-16T11:00:22.914 INFO:teuthology.orchestra.run.smithi162.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-16T11:00:22.914 INFO:teuthology.orchestra.run.smithi162.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-16T11:00:22.970 INFO:teuthology.orchestra.run.smithi162.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-16T11:00:22.970 INFO:teuthology.orchestra.run.smithi162.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-16T11:00:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-16T11:00:22.972 INFO:teuthology.orchestra.run.smithi162.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-16T11:00:23.051 INFO:teuthology.orchestra.run.smithi162.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-16T11:00:23.052 INFO:teuthology.orchestra.run.smithi162.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-16T11:00:23.052 INFO:teuthology.orchestra.run.smithi162.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-16T11:00:26.402 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 21.8 MB in 3s (7,239 kB/s) 2024-09-16T11:00:26.948 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 21.8 MB in 4s (6,137 kB/s) 2024-09-16T11:00:27.892 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-16T11:00:27.918 DEBUG:teuthology.orchestra.run.smithi165:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T11:00:27.988 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-16T11:00:28.182 INFO:teuthology.orchestra.run.smithi165.stdout:Building dependency tree... 2024-09-16T11:00:28.182 INFO:teuthology.orchestra.run.smithi165.stdout:Reading state information... 2024-09-16T11:00:28.377 INFO:teuthology.orchestra.run.smithi165.stdout:The following additional packages will be installed: 2024-09-16T11:00:28.378 INFO:teuthology.orchestra.run.smithi165.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T11:00:28.378 INFO:teuthology.orchestra.run.smithi165.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-16T11:00:28.378 INFO:teuthology.orchestra.run.smithi165.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-16T11:00:28.379 INFO:teuthology.orchestra.run.smithi165.stdout:Suggested packages: 2024-09-16T11:00:28.379 INFO:teuthology.orchestra.run.smithi165.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-16T11:00:28.379 INFO:teuthology.orchestra.run.smithi165.stdout:Recommended packages: 2024-09-16T11:00:28.379 INFO:teuthology.orchestra.run.smithi165.stdout: thermald 2024-09-16T11:00:28.442 INFO:teuthology.orchestra.run.smithi165.stdout:The following NEW packages will be installed: 2024-09-16T11:00:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T11:00:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-16T11:00:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-16T11:00:28.443 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages will be upgraded: 2024-09-16T11:00:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T11:00:28.478 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-09-16T11:00:28.504 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T11:00:28.575 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-09-16T11:00:28.624 INFO:teuthology.orchestra.run.smithi165.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T11:00:28.625 INFO:teuthology.orchestra.run.smithi165.stdout:Need to get 77.2 MB of archives. 2024-09-16T11:00:28.625 INFO:teuthology.orchestra.run.smithi165.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-16T11:00:28.625 INFO:teuthology.orchestra.run.smithi165.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-16T11:00:28.793 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-09-16T11:00:28.793 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-09-16T11:00:28.991 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2024-09-16T11:00:28.991 INFO:teuthology.orchestra.run.smithi162.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T11:00:28.991 INFO:teuthology.orchestra.run.smithi162.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-16T11:00:28.991 INFO:teuthology.orchestra.run.smithi162.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-16T11:00:28.992 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2024-09-16T11:00:28.992 INFO:teuthology.orchestra.run.smithi162.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-16T11:00:28.992 INFO:teuthology.orchestra.run.smithi162.stdout:Recommended packages: 2024-09-16T11:00:28.993 INFO:teuthology.orchestra.run.smithi162.stdout: thermald 2024-09-16T11:00:29.053 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2024-09-16T11:00:29.053 INFO:teuthology.orchestra.run.smithi162.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-16T11:00:29.053 INFO:teuthology.orchestra.run.smithi162.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-16T11:00:29.053 INFO:teuthology.orchestra.run.smithi162.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-16T11:00:29.055 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages will be upgraded: 2024-09-16T11:00:29.055 INFO:teuthology.orchestra.run.smithi162.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T11:00:29.108 INFO:teuthology.orchestra.run.smithi162.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T11:00:29.108 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 77.2 MB of archives. 2024-09-16T11:00:29.108 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-16T11:00:29.108 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-16T11:00:29.440 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-16T11:00:29.531 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-16T11:00:29.691 INFO:teuthology.orchestra.run.smithi165.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-16T11:00:29.918 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-16T11:00:30.011 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-16T11:00:30.012 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-16T11:00:30.012 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-16T11:00:30.106 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-16T11:00:30.116 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-16T11:00:30.402 INFO:teuthology.orchestra.run.smithi165.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-16T11:00:30.420 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-16T11:00:30.438 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-16T11:00:30.553 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-16T11:00:30.587 INFO:teuthology.orchestra.run.smithi165.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-16T11:00:30.755 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 77.2 MB in 1s (73.2 MB/s) 2024-09-16T11:00:30.938 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-16T11:00:31.245 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 77.2 MB in 2s (36.1 MB/s) 2024-09-16T11:00:31.428 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-16T11:00:32.617 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-16T11:00:32.622 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:32.752 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:33.159 INFO:teuthology.orchestra.run.smithi165.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-16T11:00:33.164 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:33.252 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:34.735 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-16T11:00:34.749 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:34.873 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:35.128 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-16T11:00:35.142 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:35.243 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:35.979 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-16T11:00:35.993 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:36.037 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:36.331 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-16T11:00:36.345 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:36.389 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:40.926 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T11:00:40.953 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T11:00:41.069 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T11:00:41.072 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T11:00:41.404 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T11:00:41.416 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T11:00:41.542 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T11:00:41.564 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T11:00:41.879 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-16T11:00:41.892 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-16T11:00:41.896 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-16T11:00:41.906 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-16T11:00:41.937 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T11:00:41.950 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T11:00:47.309 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-16T11:00:47.330 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:47.367 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:47.824 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-16T11:00:47.846 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-16T11:00:47.941 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:50.131 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T11:00:50.243 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T11:00:50.557 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:50.579 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-16T11:00:50.699 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-16T11:00:51.058 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:51.764 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:52.321 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-16T11:00:52.908 INFO:teuthology.orchestra.run.smithi165.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-16T11:00:53.050 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:53.730 INFO:teuthology.orchestra.run.smithi162.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-16T11:00:53.730 INFO:teuthology.orchestra.run.smithi162.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-16T11:00:53.881 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:54.226 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T11:00:54.352 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-16T11:00:54.478 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:55.057 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-16T11:00:55.175 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-16T11:00:55.293 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:55.444 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-16T11:00:55.570 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-16T11:00:55.587 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-16T11:00:55.696 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:55.770 INFO:teuthology.orchestra.run.smithi162.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T11:00:55.770 INFO:teuthology.orchestra.run.smithi162.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-16T11:00:56.472 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-16T11:00:56.598 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-16T11:00:56.668 INFO:teuthology.orchestra.run.smithi165.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T11:00:56.668 INFO:teuthology.orchestra.run.smithi165.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-16T11:01:17.850 INFO:teuthology.orchestra.run.smithi162.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T11:01:17.851 INFO:teuthology.orchestra.run.smithi162.stdout:Sourcing file `/etc/default/grub' 2024-09-16T11:01:17.860 INFO:teuthology.orchestra.run.smithi162.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T11:01:17.870 INFO:teuthology.orchestra.run.smithi162.stdout:Generating grub configuration file ... 2024-09-16T11:01:18.202 INFO:teuthology.orchestra.run.smithi162.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T11:01:18.221 INFO:teuthology.orchestra.run.smithi162.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T11:01:18.538 INFO:teuthology.orchestra.run.smithi165.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T11:01:18.539 INFO:teuthology.orchestra.run.smithi165.stdout:Sourcing file `/etc/default/grub' 2024-09-16T11:01:18.559 INFO:teuthology.orchestra.run.smithi165.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T11:01:18.568 INFO:teuthology.orchestra.run.smithi165.stdout:Generating grub configuration file ... 2024-09-16T11:01:18.617 INFO:teuthology.orchestra.run.smithi162.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T11:01:18.628 INFO:teuthology.orchestra.run.smithi162.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T11:01:18.914 INFO:teuthology.orchestra.run.smithi165.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T11:01:18.936 INFO:teuthology.orchestra.run.smithi165.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T11:01:19.346 INFO:teuthology.orchestra.run.smithi165.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T11:01:19.357 INFO:teuthology.orchestra.run.smithi165.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T11:01:20.463 INFO:teuthology.orchestra.run.smithi162.stdout:done 2024-09-16T11:01:20.888 DEBUG:teuthology.orchestra.run.smithi162:> dpkg -s linux-image-generic 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Package: linux-image-generic 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Status: install ok installed 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Priority: optional 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Section: kernel 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Installed-Size: 20 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T11:01:20.935 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: amd64 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:Source: linux-meta 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:Version: 5.4.0.195.193 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:Recommends: thermald 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:Description: Generic Linux kernel image 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: This package will always depend on the latest generic kernel image 2024-09-16T11:01:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: available. 2024-09-16T11:01:20.937 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-16T11:01:20.937 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T11:01:20.937 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T11:01:20.938 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T11:01:20.938 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T11:01:20.938 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get clean 2024-09-16T11:01:21.035 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get update 2024-09-16T11:01:21.165 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T11:01:21.165 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T11:01:21.172 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T11:01:21.189 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T11:01:21.387 INFO:teuthology.orchestra.run.smithi165.stdout:done 2024-09-16T11:01:21.803 DEBUG:teuthology.orchestra.run.smithi165:> dpkg -s linux-image-generic 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Package: linux-image-generic 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Status: install ok installed 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Priority: optional 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Section: kernel 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Installed-Size: 20 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: amd64 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Source: linux-meta 2024-09-16T11:01:21.856 INFO:teuthology.orchestra.run.smithi165.stdout:Version: 5.4.0.195.193 2024-09-16T11:01:21.857 INFO:teuthology.orchestra.run.smithi165.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T11:01:21.857 INFO:teuthology.orchestra.run.smithi165.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T11:01:21.857 INFO:teuthology.orchestra.run.smithi165.stdout:Recommends: thermald 2024-09-16T11:01:21.857 INFO:teuthology.orchestra.run.smithi165.stdout:Description: Generic Linux kernel image 2024-09-16T11:01:21.857 INFO:teuthology.orchestra.run.smithi165.stdout: This package will always depend on the latest generic kernel image 2024-09-16T11:01:21.857 INFO:teuthology.orchestra.run.smithi165.stdout: available. 2024-09-16T11:01:21.857 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-16T11:01:21.858 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T11:01:21.858 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T11:01:21.858 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T11:01:21.858 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi165.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T11:01:21.858 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get clean 2024-09-16T11:01:21.950 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get update 2024-09-16T11:01:22.211 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T11:01:22.211 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T11:01:22.282 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T11:01:22.364 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T11:01:23.030 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-09-16T11:01:23.056 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T11:01:23.126 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-09-16T11:01:23.344 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-09-16T11:01:23.345 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-09-16T11:01:23.584 INFO:teuthology.orchestra.run.smithi162.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-16T11:01:23.584 INFO:teuthology.orchestra.run.smithi162.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T11:01:23.586 DEBUG:teuthology.orchestra.run.smithi162:> dpkg -s linux-image-generic 2024-09-16T11:01:23.606 INFO:teuthology.orchestra.run.smithi162.stdout:Package: linux-image-generic 2024-09-16T11:01:23.606 INFO:teuthology.orchestra.run.smithi162.stdout:Status: install ok installed 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Priority: optional 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Section: kernel 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Installed-Size: 20 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: amd64 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Source: linux-meta 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Version: 5.4.0.195.193 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Recommends: thermald 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout:Description: Generic Linux kernel image 2024-09-16T11:01:23.607 INFO:teuthology.orchestra.run.smithi162.stdout: This package will always depend on the latest generic kernel image 2024-09-16T11:01:23.608 INFO:teuthology.orchestra.run.smithi162.stdout: available. 2024-09-16T11:01:23.608 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T11:01:23.613 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.IUmgUeFE7Q 2024-09-16T11:01:23.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub/grub.cfg /tmp/tmp.IUmgUeFE7Q 2024-09-16T11:01:23.626 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.IUmgUeFE7Q 2024-09-16T11:01:23.708 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.IUmgUeFE7Q is 9KB 2024-09-16T11:01:23.720 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.IUmgUeFE7Q 2024-09-16T11:01:23.727 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T11:01:23.782 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:01:23.782 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T11:01:23.783 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T11:01:23.813 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-16T11:01:23.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo update-grub 2024-09-16T11:01:24.071 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-16T11:01:24.099 DEBUG:teuthology.orchestra.run.smithi165:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T11:01:24.167 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-16T11:01:24.382 INFO:teuthology.orchestra.run.smithi165.stdout:Building dependency tree... 2024-09-16T11:01:24.382 INFO:teuthology.orchestra.run.smithi165.stdout:Reading state information... 2024-09-16T11:01:24.611 INFO:teuthology.orchestra.run.smithi165.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-16T11:01:24.611 INFO:teuthology.orchestra.run.smithi165.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-16T11:01:24.613 DEBUG:teuthology.orchestra.run.smithi165:> dpkg -s linux-image-generic 2024-09-16T11:01:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:Package: linux-image-generic 2024-09-16T11:01:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:Status: install ok installed 2024-09-16T11:01:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:Priority: optional 2024-09-16T11:01:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:Section: kernel 2024-09-16T11:01:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:Installed-Size: 20 2024-09-16T11:01:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: amd64 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Source: linux-meta 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Version: 5.4.0.195.193 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Recommends: thermald 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout:Description: Generic Linux kernel image 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout: This package will always depend on the latest generic kernel image 2024-09-16T11:01:24.634 INFO:teuthology.orchestra.run.smithi165.stdout: available. 2024-09-16T11:01:24.635 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-09-16T11:01:24.683 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.iBITjmaI2G 2024-09-16T11:01:24.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /boot/grub/grub.cfg /tmp/tmp.iBITjmaI2G 2024-09-16T11:01:24.698 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.iBITjmaI2G 2024-09-16T11:01:24.734 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.iBITjmaI2G is 9KB 2024-09-16T11:01:24.748 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.iBITjmaI2G 2024-09-16T11:01:24.754 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T11:01:24.810 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:01:24.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T11:01:24.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T11:01:24.836 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-16T11:01:24.836 DEBUG:teuthology.orchestra.run.smithi165:> sudo update-grub 2024-09-16T11:01:25.158 INFO:teuthology.orchestra.run.smithi162.stderr:Sourcing file `/etc/default/grub' 2024-09-16T11:01:25.167 INFO:teuthology.orchestra.run.smithi162.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T11:01:25.183 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2024-09-16T11:01:25.424 INFO:teuthology.orchestra.run.smithi162.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T11:01:25.436 INFO:teuthology.orchestra.run.smithi162.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T11:01:25.812 INFO:teuthology.orchestra.run.smithi162.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T11:01:25.824 INFO:teuthology.orchestra.run.smithi162.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T11:01:26.162 INFO:teuthology.orchestra.run.smithi165.stderr:Sourcing file `/etc/default/grub' 2024-09-16T11:01:26.174 INFO:teuthology.orchestra.run.smithi165.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T11:01:26.185 INFO:teuthology.orchestra.run.smithi165.stderr:Generating grub configuration file ... 2024-09-16T11:01:26.412 INFO:teuthology.orchestra.run.smithi165.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-16T11:01:26.424 INFO:teuthology.orchestra.run.smithi165.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-16T11:01:26.578 INFO:teuthology.orchestra.run.smithi162.stderr:done 2024-09-16T11:01:26.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2024-09-16T11:01:26.801 INFO:teuthology.orchestra.run.smithi165.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-16T11:01:26.812 INFO:teuthology.orchestra.run.smithi165.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-16T11:01:27.802 INFO:teuthology.orchestra.run.smithi165.stderr:done 2024-09-16T11:01:27.804 DEBUG:teuthology.orchestra.run.smithi165:> sudo shutdown -r now 2024-09-16T11:01:56.609 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T11:01:56.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:01:56.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:01:57.808 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T11:01:57.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:01:57.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:02:15.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-16T11:02:24.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:02:24.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:02:27.165 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-16T11:02:32.253 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-09-16T11:02:39.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:02:39.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:02:41.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:02:41.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:02:59.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-09-16T11:03:11.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:03:11.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:03:11.660 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-09-16T11:03:26.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:03:26.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:03:26.922 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-16T11:03:27.527 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:03:27.527 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-16T11:03:27.528 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-09-16T11:03:27.575 INFO:teuthology.orchestra.run.smithi165.stdout:5.4.0-195-generic 2024-09-16T11:03:27.576 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-16T11:03:27.576 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T11:03:27.576 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T11:03:28.576 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T11:03:28.577 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T11:03:28.665 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2024-09-16T11:03:28.684 DEBUG:teuthology.parallel:result is None 2024-09-16T11:03:39.176 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T11:03:54.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:03:54.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:03:54.445 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-16T11:03:55.048 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:03:55.048 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-16T11:03:55.048 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-16T11:03:55.097 INFO:teuthology.orchestra.run.smithi162.stdout:5.4.0-195-generic 2024-09-16T11:03:55.097 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-16T11:03:55.097 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T11:03:55.097 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T11:03:56.098 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T11:03:56.098 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T11:03:56.189 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2024-09-16T11:03:56.234 DEBUG:teuthology.parallel:result is None 2024-09-16T11:03:56.234 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T11:03:56.242 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T11:03:56.242 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T11:03:56.245 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T11:03:56.252 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T11:03:56.295 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T11:03:56.324 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T11:03:56.325 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T11:03:56.327 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T11:03:56.349 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T11:03:56.355 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T11:03:56.356 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T11:03:56.358 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T11:03:56.372 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:03:56.378 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:03:56.383 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:03:56.389 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T11:03:56.391 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T11:03:56.398 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T11:03:56.398 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T11:03:56.429 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T11:03:56.445 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T11:03:56.518 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T11:03:56.519 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T11:03:56.521 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T11:03:56.528 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T11:03:56.534 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T11:03:56.582 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:03:56.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T11:03:56.603 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T11:03:56.610 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T11:03:56.617 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:03:56.617 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T11:03:56.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-09-16T11:03:56.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-09-16T11:03:56.733 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T11:03:56.741 INFO:teuthology.task.internal:Starting timer... 2024-09-16T11:03:56.741 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T11:03:56.763 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T11:03:56.792 DEBUG:teuthology.task.selinux:Excluding smithi162: OS 'ubuntu' does not support SELinux 2024-09-16T11:03:56.792 DEBUG:teuthology.task.selinux:Excluding smithi165: OS 'ubuntu' does not support SELinux 2024-09-16T11:03:56.792 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T11:03:56.792 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-16T11:03:56.792 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T11:03:56.793 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T11:03:56.802 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T11:03:56.816 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T11:03:56.817 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi162.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T11:09:38.181 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi162.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-09-16T11:09:38.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:09:38.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:09:38.300 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-16T11:09:38.426 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-16T11:09:38.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:09:38.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T11:09:38.515 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-16T11:09:38.643 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-16T11:09:38.643 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T11:09:38.654 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T11:09:38.654 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T11:09:38.654 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:09:38.657 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T11:09:38.657 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:09:38.689 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-16T11:09:38.689 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Command line: ntpd -gq 2024-09-16T11:09:38.690 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: proto: precision = 0.154 usec (-23) 2024-09-16T11:09:38.691 INFO:teuthology.orchestra.run.smithi162.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T11:09:38.691 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T11:09:38.692 INFO:teuthology.orchestra.run.smithi162.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listen normally on 3 enp3s0f1 172.21.15.162:123 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listen normally on 4 lo [::1]:123 2024-09-16T11:09:38.694 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f41%5]:123 2024-09-16T11:09:38.695 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:38 ntpd[13770]: Listening on routing socket on fd #22 for interface updates 2024-09-16T11:09:38.719 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-16T11:09:38.719 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Command line: ntpd -gq 2024-09-16T11:09:38.719 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: proto: precision = 0.235 usec (-22) 2024-09-16T11:09:38.720 INFO:teuthology.orchestra.run.smithi165.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T11:09:38.720 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T11:09:38.720 INFO:teuthology.orchestra.run.smithi165.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T11:09:38.721 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listen normally on 3 enp3s0f1 172.21.15.165:123 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listen normally on 4 lo [::1]:123 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f63%5]:123 2024-09-16T11:09:38.722 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:38 ntpd[13764]: Listening on routing socket on fd #22 for interface updates 2024-09-16T11:09:39.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:39.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:39 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:39.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:39.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:39 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:40 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:40 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:40 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:40 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:40 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:40.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:40 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:41.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:41 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:41 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:41 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:41 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:41 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:41.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:41 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:41.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:41.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:41.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:41.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:41.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:41.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:41 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:42.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:42.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:42.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:42.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:42.693 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:42.694 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:42 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:43.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:43 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:43 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:43 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:43 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:43 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:43.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:43 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:43.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:43.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:43.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:43.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:43.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:43.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:43 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:44.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:44.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:44.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:44.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:44.692 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:44 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:45.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.690 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:45.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:45.691 INFO:teuthology.orchestra.run.smithi162.stdout:16 Sep 11:09:45 ntpd[13770]: ntpd: time slew +0.000488 s 2024-09-16T11:09:45.691 INFO:teuthology.orchestra.run.smithi162.stdout:ntpd: time slew +0.000488s 2024-09-16T11:09:45.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T11:09:45.691 INFO:teuthology.orchestra.run.smithi162.stderr:16 Sep 11:09:45 ntpd[13770]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.720 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stdout:16 Sep 11:09:45 ntpd[13764]: ntpd: time slew -0.005819 s 2024-09-16T11:09:45.721 INFO:teuthology.orchestra.run.smithi165.stdout:ntpd: time slew -0.005819s 2024-09-16T11:09:45.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T11:09:45.722 INFO:teuthology.orchestra.run.smithi165.stderr:16 Sep 11:09:45 ntpd[13764]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T11:09:45.766 INFO:teuthology.orchestra.run.smithi162.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T11:09:45.766 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================== 2024-09-16T11:09:45.766 INFO:teuthology.orchestra.run.smithi162.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.766 INFO:teuthology.orchestra.run.smithi162.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.767 INFO:teuthology.orchestra.run.smithi162.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.767 INFO:teuthology.orchestra.run.smithi162.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.784 INFO:teuthology.orchestra.run.smithi165.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T11:09:45.785 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================== 2024-09-16T11:09:45.785 INFO:teuthology.orchestra.run.smithi165.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.785 INFO:teuthology.orchestra.run.smithi165.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.785 INFO:teuthology.orchestra.run.smithi165.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.785 INFO:teuthology.orchestra.run.smithi165.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:09:45.786 INFO:teuthology.run_tasks:Running task install... 2024-09-16T11:09:45.794 DEBUG:teuthology.task.install:project ceph 2024-09-16T11:09:45.794 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T11:09:45.795 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-16T11:09:45.795 INFO:teuthology.task.install:Using flavor: default 2024-09-16T11:09:45.801 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T11:09:45.801 INFO:teuthology.task.install:extra packages: [] 2024-09-16T11:09:45.801 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-key list | grep Ceph 2024-09-16T11:09:45.802 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-key list | grep Ceph 2024-09-16T11:09:45.904 INFO:teuthology.orchestra.run.smithi162.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-16T11:09:45.905 INFO:teuthology.orchestra.run.smithi165.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-16T11:09:45.965 INFO:teuthology.orchestra.run.smithi165.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T11:09:45.966 INFO:teuthology.orchestra.run.smithi165.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T11:09:45.967 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T11:09:45.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:09:45.972 INFO:teuthology.orchestra.run.smithi162.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T11:09:45.972 INFO:teuthology.orchestra.run.smithi162.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T11:09:45.973 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T11:09:45.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:09:46.110 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-16T11:09:46.110 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-16T11:09:46.129 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-16T11:09:46.129 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-16T11:09:46.221 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:09:46.221 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T11:09:46.238 DEBUG:teuthology.orchestra.run.smithi165:> sudo apt-get update 2024-09-16T11:09:46.240 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:09:46.240 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T11:09:46.256 DEBUG:teuthology.orchestra.run.smithi162:> sudo apt-get update 2024-09-16T11:09:46.421 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T11:09:46.443 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T11:09:46.450 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T11:09:46.467 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T11:09:46.476 INFO:teuthology.orchestra.run.smithi165.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-16T11:09:46.497 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-16T11:09:46.499 INFO:teuthology.orchestra.run.smithi162.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-16T11:09:46.520 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-16T11:09:46.521 INFO:teuthology.orchestra.run.smithi165.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-16T11:09:46.540 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-16T11:09:46.542 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-16T11:09:46.543 INFO:teuthology.orchestra.run.smithi162.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-16T11:09:46.561 INFO:teuthology.orchestra.run.smithi162.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-16T11:09:46.564 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-16T11:09:46.575 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-16T11:09:46.601 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-16T11:09:46.622 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-16T11:09:46.703 INFO:teuthology.orchestra.run.smithi165.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-16T11:09:47.076 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 28.5 kB in 1s (45.4 kB/s) 2024-09-16T11:09:47.111 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 28.5 kB in 1s (44.1 kB/s) 2024-09-16T11:09:48.227 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-16T11:09:48.254 DEBUG:teuthology.orchestra.run.smithi165:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-16T11:09:48.255 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-09-16T11:09:48.282 DEBUG:teuthology.orchestra.run.smithi162:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-16T11:09:48.327 INFO:teuthology.orchestra.run.smithi165.stdout:Reading package lists... 2024-09-16T11:09:48.354 INFO:teuthology.orchestra.run.smithi162.stdout:Reading package lists... 2024-09-16T11:09:48.528 INFO:teuthology.orchestra.run.smithi165.stdout:Building dependency tree... 2024-09-16T11:09:48.529 INFO:teuthology.orchestra.run.smithi165.stdout:Reading state information... 2024-09-16T11:09:48.546 INFO:teuthology.orchestra.run.smithi162.stdout:Building dependency tree... 2024-09-16T11:09:48.547 INFO:teuthology.orchestra.run.smithi162.stdout:Reading state information... 2024-09-16T11:09:48.730 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout:The following additional packages will be installed: 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-16T11:09:48.732 INFO:teuthology.orchestra.run.smithi165.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-16T11:09:48.733 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-16T11:09:48.733 INFO:teuthology.orchestra.run.smithi165.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-16T11:09:48.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-16T11:09:48.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-16T11:09:48.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-16T11:09:48.733 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-16T11:09:48.734 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: socat xmlstarlet 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout:Suggested packages: 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-16T11:09:48.735 INFO:teuthology.orchestra.run.smithi165.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-16T11:09:48.736 INFO:teuthology.orchestra.run.smithi165.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T11:09:48.736 INFO:teuthology.orchestra.run.smithi165.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-16T11:09:48.736 INFO:teuthology.orchestra.run.smithi165.stdout:Recommended packages: 2024-09-16T11:09:48.736 INFO:teuthology.orchestra.run.smithi165.stdout: btrfs-tools 2024-09-16T11:09:48.741 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout:The following additional packages will be installed: 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-16T11:09:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-16T11:09:48.743 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T11:09:48.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout: socat xmlstarlet 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout:Suggested packages: 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-16T11:09:48.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout:Recommended packages: 2024-09-16T11:09:48.746 INFO:teuthology.orchestra.run.smithi162.stdout: btrfs-tools 2024-09-16T11:09:48.853 INFO:teuthology.orchestra.run.smithi165.stdout:The following NEW packages will be installed: 2024-09-16T11:09:48.853 INFO:teuthology.orchestra.run.smithi165.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T11:09:48.853 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T11:09:48.853 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-16T11:09:48.854 INFO:teuthology.orchestra.run.smithi165.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-16T11:09:48.854 INFO:teuthology.orchestra.run.smithi165.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-16T11:09:48.854 INFO:teuthology.orchestra.run.smithi165.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-16T11:09:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-16T11:09:48.856 INFO:teuthology.orchestra.run.smithi165.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-16T11:09:48.856 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-16T11:09:48.856 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-16T11:09:48.856 INFO:teuthology.orchestra.run.smithi165.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-16T11:09:48.856 INFO:teuthology.orchestra.run.smithi165.stdout:The following packages will be upgraded: 2024-09-16T11:09:48.856 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 librbd1 2024-09-16T11:09:48.863 INFO:teuthology.orchestra.run.smithi162.stdout:The following NEW packages will be installed: 2024-09-16T11:09:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T11:09:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T11:09:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-16T11:09:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-16T11:09:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-16T11:09:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-16T11:09:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-16T11:09:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-16T11:09:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-16T11:09:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-16T11:09:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-16T11:09:48.867 INFO:teuthology.orchestra.run.smithi162.stdout:The following packages will be upgraded: 2024-09-16T11:09:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 librbd1 2024-09-16T11:09:48.931 INFO:teuthology.orchestra.run.smithi162.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-16T11:09:48.931 INFO:teuthology.orchestra.run.smithi162.stdout:Need to get 201 MB of archives. 2024-09-16T11:09:48.931 INFO:teuthology.orchestra.run.smithi162.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-16T11:09:48.931 INFO:teuthology.orchestra.run.smithi162.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-16T11:09:49.010 INFO:teuthology.orchestra.run.smithi165.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-16T11:09:49.010 INFO:teuthology.orchestra.run.smithi165.stdout:Need to get 201 MB of archives. 2024-09-16T11:09:49.010 INFO:teuthology.orchestra.run.smithi165.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-16T11:09:49.011 INFO:teuthology.orchestra.run.smithi165.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-16T11:09:49.015 INFO:teuthology.orchestra.run.smithi162.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-16T11:09:49.016 INFO:teuthology.orchestra.run.smithi162.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-16T11:09:49.026 INFO:teuthology.orchestra.run.smithi162.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-16T11:09:49.027 INFO:teuthology.orchestra.run.smithi162.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-16T11:09:49.032 INFO:teuthology.orchestra.run.smithi162.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-16T11:09:49.049 INFO:teuthology.orchestra.run.smithi165.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-16T11:09:49.057 INFO:teuthology.orchestra.run.smithi162.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-16T11:09:49.058 INFO:teuthology.orchestra.run.smithi162.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-16T11:09:49.064 INFO:teuthology.orchestra.run.smithi162.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-16T11:09:49.106 INFO:teuthology.orchestra.run.smithi162.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-16T11:09:49.108 INFO:teuthology.orchestra.run.smithi162.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-16T11:09:49.120 INFO:teuthology.orchestra.run.smithi162.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-16T11:09:49.122 INFO:teuthology.orchestra.run.smithi162.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-16T11:09:49.122 INFO:teuthology.orchestra.run.smithi162.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-16T11:09:49.125 INFO:teuthology.orchestra.run.smithi162.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-16T11:09:49.125 INFO:teuthology.orchestra.run.smithi162.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-16T11:09:49.126 INFO:teuthology.orchestra.run.smithi162.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-16T11:09:49.126 INFO:teuthology.orchestra.run.smithi162.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-16T11:09:49.135 INFO:teuthology.orchestra.run.smithi162.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-16T11:09:49.156 INFO:teuthology.orchestra.run.smithi162.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-16T11:09:49.156 INFO:teuthology.orchestra.run.smithi162.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-16T11:09:49.157 INFO:teuthology.orchestra.run.smithi162.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-16T11:09:49.158 INFO:teuthology.orchestra.run.smithi162.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-16T11:09:49.159 INFO:teuthology.orchestra.run.smithi162.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T11:09:49.160 INFO:teuthology.orchestra.run.smithi162.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-16T11:09:49.161 INFO:teuthology.orchestra.run.smithi162.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-16T11:09:49.163 INFO:teuthology.orchestra.run.smithi162.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-16T11:09:49.164 INFO:teuthology.orchestra.run.smithi162.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-16T11:09:49.179 INFO:teuthology.orchestra.run.smithi162.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-16T11:09:49.181 INFO:teuthology.orchestra.run.smithi162.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-16T11:09:49.182 INFO:teuthology.orchestra.run.smithi162.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-16T11:09:49.187 INFO:teuthology.orchestra.run.smithi162.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-16T11:09:49.188 INFO:teuthology.orchestra.run.smithi162.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-16T11:09:49.197 INFO:teuthology.orchestra.run.smithi162.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-16T11:09:49.197 INFO:teuthology.orchestra.run.smithi162.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-16T11:09:49.198 INFO:teuthology.orchestra.run.smithi162.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-16T11:09:49.201 INFO:teuthology.orchestra.run.smithi162.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-16T11:09:49.201 INFO:teuthology.orchestra.run.smithi162.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T11:09:49.214 INFO:teuthology.orchestra.run.smithi162.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-16T11:09:49.216 INFO:teuthology.orchestra.run.smithi162.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-16T11:09:49.217 INFO:teuthology.orchestra.run.smithi162.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-16T11:09:49.219 INFO:teuthology.orchestra.run.smithi162.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-16T11:09:49.220 INFO:teuthology.orchestra.run.smithi162.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-16T11:09:49.232 INFO:teuthology.orchestra.run.smithi162.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-16T11:09:49.312 INFO:teuthology.orchestra.run.smithi162.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-16T11:09:49.330 INFO:teuthology.orchestra.run.smithi162.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-16T11:09:49.331 INFO:teuthology.orchestra.run.smithi162.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-16T11:09:49.343 INFO:teuthology.orchestra.run.smithi162.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-16T11:09:49.348 INFO:teuthology.orchestra.run.smithi162.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-16T11:09:49.348 INFO:teuthology.orchestra.run.smithi162.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-16T11:09:49.410 INFO:teuthology.orchestra.run.smithi162.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-16T11:09:49.411 INFO:teuthology.orchestra.run.smithi162.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-16T11:09:49.412 INFO:teuthology.orchestra.run.smithi162.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-16T11:09:49.413 INFO:teuthology.orchestra.run.smithi162.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-16T11:09:49.415 INFO:teuthology.orchestra.run.smithi162.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-16T11:09:49.416 INFO:teuthology.orchestra.run.smithi162.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-16T11:09:49.417 INFO:teuthology.orchestra.run.smithi162.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-16T11:09:49.424 INFO:teuthology.orchestra.run.smithi162.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-16T11:09:49.425 INFO:teuthology.orchestra.run.smithi162.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-16T11:09:49.429 INFO:teuthology.orchestra.run.smithi162.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-16T11:09:49.430 INFO:teuthology.orchestra.run.smithi162.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-16T11:09:49.447 INFO:teuthology.orchestra.run.smithi162.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-16T11:09:49.449 INFO:teuthology.orchestra.run.smithi162.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-16T11:09:49.466 INFO:teuthology.orchestra.run.smithi165.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-16T11:09:49.467 INFO:teuthology.orchestra.run.smithi162.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-16T11:09:49.470 INFO:teuthology.orchestra.run.smithi162.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-16T11:09:49.475 INFO:teuthology.orchestra.run.smithi165.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-16T11:09:49.510 INFO:teuthology.orchestra.run.smithi162.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-16T11:09:49.511 INFO:teuthology.orchestra.run.smithi162.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-16T11:09:49.511 INFO:teuthology.orchestra.run.smithi162.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-16T11:09:49.512 INFO:teuthology.orchestra.run.smithi162.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-16T11:09:49.512 INFO:teuthology.orchestra.run.smithi162.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-16T11:09:49.513 INFO:teuthology.orchestra.run.smithi162.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-16T11:09:49.523 INFO:teuthology.orchestra.run.smithi165.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-16T11:09:49.524 INFO:teuthology.orchestra.run.smithi162.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T11:09:49.524 INFO:teuthology.orchestra.run.smithi162.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-16T11:09:49.524 INFO:teuthology.orchestra.run.smithi162.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-16T11:09:49.528 INFO:teuthology.orchestra.run.smithi162.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-16T11:09:49.545 INFO:teuthology.orchestra.run.smithi162.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-16T11:09:49.545 INFO:teuthology.orchestra.run.smithi162.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-16T11:09:49.546 INFO:teuthology.orchestra.run.smithi162.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-16T11:09:49.548 INFO:teuthology.orchestra.run.smithi162.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-16T11:09:49.549 INFO:teuthology.orchestra.run.smithi162.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-16T11:09:49.550 INFO:teuthology.orchestra.run.smithi165.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-16T11:09:49.662 INFO:teuthology.orchestra.run.smithi165.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-16T11:09:49.668 INFO:teuthology.orchestra.run.smithi165.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-16T11:09:49.698 INFO:teuthology.orchestra.run.smithi165.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-16T11:09:49.839 INFO:teuthology.orchestra.run.smithi165.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-16T11:09:49.847 INFO:teuthology.orchestra.run.smithi165.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-16T11:09:49.869 INFO:teuthology.orchestra.run.smithi165.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-16T11:09:49.877 INFO:teuthology.orchestra.run.smithi165.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-16T11:09:49.878 INFO:teuthology.orchestra.run.smithi165.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-16T11:09:49.885 INFO:teuthology.orchestra.run.smithi165.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-16T11:09:49.886 INFO:teuthology.orchestra.run.smithi165.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-16T11:09:49.887 INFO:teuthology.orchestra.run.smithi165.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-16T11:09:49.887 INFO:teuthology.orchestra.run.smithi165.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-16T11:09:49.921 INFO:teuthology.orchestra.run.smithi165.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-16T11:09:49.973 INFO:teuthology.orchestra.run.smithi165.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-16T11:09:49.974 INFO:teuthology.orchestra.run.smithi165.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-16T11:09:49.975 INFO:teuthology.orchestra.run.smithi165.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-16T11:09:49.976 INFO:teuthology.orchestra.run.smithi165.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-16T11:09:49.978 INFO:teuthology.orchestra.run.smithi165.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T11:09:49.979 INFO:teuthology.orchestra.run.smithi165.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-16T11:09:49.980 INFO:teuthology.orchestra.run.smithi165.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-16T11:09:49.982 INFO:teuthology.orchestra.run.smithi165.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-16T11:09:50.002 INFO:teuthology.orchestra.run.smithi165.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-16T11:09:50.065 INFO:teuthology.orchestra.run.smithi165.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-16T11:09:50.067 INFO:teuthology.orchestra.run.smithi165.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-16T11:09:50.069 INFO:teuthology.orchestra.run.smithi165.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-16T11:09:50.077 INFO:teuthology.orchestra.run.smithi165.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-16T11:09:50.077 INFO:teuthology.orchestra.run.smithi165.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-16T11:09:50.077 INFO:teuthology.orchestra.run.smithi165.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-16T11:09:50.077 INFO:teuthology.orchestra.run.smithi165.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-16T11:09:50.085 INFO:teuthology.orchestra.run.smithi165.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-16T11:09:50.090 INFO:teuthology.orchestra.run.smithi165.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-16T11:09:50.147 INFO:teuthology.orchestra.run.smithi165.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-16T11:09:50.165 INFO:teuthology.orchestra.run.smithi165.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T11:09:50.166 INFO:teuthology.orchestra.run.smithi165.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-16T11:09:50.168 INFO:teuthology.orchestra.run.smithi165.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-16T11:09:50.171 INFO:teuthology.orchestra.run.smithi165.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-16T11:09:50.173 INFO:teuthology.orchestra.run.smithi165.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-16T11:09:50.174 INFO:teuthology.orchestra.run.smithi165.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-16T11:09:50.175 INFO:teuthology.orchestra.run.smithi165.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-16T11:09:50.422 INFO:teuthology.orchestra.run.smithi162.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-16T11:09:50.497 INFO:teuthology.orchestra.run.smithi165.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-16T11:09:50.524 INFO:teuthology.orchestra.run.smithi165.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-16T11:09:50.527 INFO:teuthology.orchestra.run.smithi165.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-16T11:09:50.597 INFO:teuthology.orchestra.run.smithi165.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-16T11:09:50.597 INFO:teuthology.orchestra.run.smithi165.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-16T11:09:50.597 INFO:teuthology.orchestra.run.smithi165.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-16T11:09:50.597 INFO:teuthology.orchestra.run.smithi165.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-16T11:09:50.598 INFO:teuthology.orchestra.run.smithi165.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-16T11:09:50.599 INFO:teuthology.orchestra.run.smithi165.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-16T11:09:50.599 INFO:teuthology.orchestra.run.smithi165.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-16T11:09:50.661 INFO:teuthology.orchestra.run.smithi165.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-16T11:09:50.662 INFO:teuthology.orchestra.run.smithi165.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-16T11:09:50.743 INFO:teuthology.orchestra.run.smithi165.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-16T11:09:50.748 INFO:teuthology.orchestra.run.smithi165.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-16T11:09:50.751 INFO:teuthology.orchestra.run.smithi165.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-16T11:09:50.754 INFO:teuthology.orchestra.run.smithi165.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-16T11:09:50.756 INFO:teuthology.orchestra.run.smithi165.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-16T11:09:50.758 INFO:teuthology.orchestra.run.smithi165.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-16T11:09:50.762 INFO:teuthology.orchestra.run.smithi165.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-16T11:09:50.835 INFO:teuthology.orchestra.run.smithi165.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-16T11:09:50.838 INFO:teuthology.orchestra.run.smithi165.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-16T11:09:50.999 INFO:teuthology.orchestra.run.smithi165.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-16T11:09:51.000 INFO:teuthology.orchestra.run.smithi165.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-16T11:09:51.001 INFO:teuthology.orchestra.run.smithi165.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-16T11:09:51.001 INFO:teuthology.orchestra.run.smithi165.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-16T11:09:51.001 INFO:teuthology.orchestra.run.smithi165.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-16T11:09:51.002 INFO:teuthology.orchestra.run.smithi165.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-16T11:09:51.016 INFO:teuthology.orchestra.run.smithi165.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T11:09:51.016 INFO:teuthology.orchestra.run.smithi165.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-16T11:09:51.041 INFO:teuthology.orchestra.run.smithi165.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-16T11:09:51.082 INFO:teuthology.orchestra.run.smithi165.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-16T11:09:51.164 INFO:teuthology.orchestra.run.smithi165.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-16T11:09:51.164 INFO:teuthology.orchestra.run.smithi165.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-16T11:09:51.165 INFO:teuthology.orchestra.run.smithi165.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-16T11:09:51.168 INFO:teuthology.orchestra.run.smithi165.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-16T11:09:51.168 INFO:teuthology.orchestra.run.smithi165.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-16T11:09:51.174 INFO:teuthology.orchestra.run.smithi165.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-16T11:09:51.360 INFO:teuthology.orchestra.run.smithi165.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-16T11:09:51.429 INFO:teuthology.orchestra.run.smithi165.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-16T11:09:51.430 INFO:teuthology.orchestra.run.smithi165.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-16T11:09:51.492 INFO:teuthology.orchestra.run.smithi165.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-16T11:09:51.493 INFO:teuthology.orchestra.run.smithi165.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-16T11:09:51.617 INFO:teuthology.orchestra.run.smithi165.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-16T11:09:51.749 INFO:teuthology.orchestra.run.smithi162.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-16T11:09:52.011 INFO:teuthology.orchestra.run.smithi162.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-16T11:09:52.145 INFO:teuthology.orchestra.run.smithi162.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-16T11:09:52.145 INFO:teuthology.orchestra.run.smithi162.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-16T11:09:52.212 INFO:teuthology.orchestra.run.smithi162.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-16T11:09:52.269 INFO:teuthology.orchestra.run.smithi162.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-16T11:09:52.405 INFO:teuthology.orchestra.run.smithi162.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-16T11:09:53.006 INFO:teuthology.orchestra.run.smithi165.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-16T11:09:53.028 INFO:teuthology.orchestra.run.smithi165.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-16T11:09:53.150 INFO:teuthology.orchestra.run.smithi165.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-16T11:09:54.351 INFO:teuthology.orchestra.run.smithi162.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-16T11:09:54.422 INFO:teuthology.orchestra.run.smithi162.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-16T11:09:54.557 INFO:teuthology.orchestra.run.smithi162.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-16T11:09:56.711 INFO:teuthology.orchestra.run.smithi165.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-16T11:09:57.009 INFO:teuthology.orchestra.run.smithi165.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-16T11:09:57.027 INFO:teuthology.orchestra.run.smithi165.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-16T11:09:57.030 INFO:teuthology.orchestra.run.smithi165.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-16T11:09:57.133 INFO:teuthology.orchestra.run.smithi165.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-16T11:09:57.436 INFO:teuthology.orchestra.run.smithi165.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-16T11:09:57.518 INFO:teuthology.orchestra.run.smithi162.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-16T11:09:57.873 INFO:teuthology.orchestra.run.smithi162.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-16T11:09:57.878 INFO:teuthology.orchestra.run.smithi162.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-16T11:09:57.895 INFO:teuthology.orchestra.run.smithi162.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-16T11:09:58.002 INFO:teuthology.orchestra.run.smithi162.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-16T11:09:58.335 INFO:teuthology.orchestra.run.smithi162.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-16T11:09:59.417 INFO:teuthology.orchestra.run.smithi165.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-16T11:09:59.418 INFO:teuthology.orchestra.run.smithi165.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-16T11:09:59.476 INFO:teuthology.orchestra.run.smithi165.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-16T11:09:59.866 INFO:teuthology.orchestra.run.smithi165.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-16T11:09:59.868 INFO:teuthology.orchestra.run.smithi165.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-16T11:09:59.871 INFO:teuthology.orchestra.run.smithi165.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-16T11:10:00.928 INFO:teuthology.orchestra.run.smithi165.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-16T11:10:02.014 INFO:teuthology.orchestra.run.smithi165.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-16T11:10:02.014 INFO:teuthology.orchestra.run.smithi165.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-16T11:10:02.138 INFO:teuthology.orchestra.run.smithi162.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-16T11:10:02.139 INFO:teuthology.orchestra.run.smithi162.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-16T11:10:02.171 INFO:teuthology.orchestra.run.smithi162.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-16T11:10:02.491 INFO:teuthology.orchestra.run.smithi162.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-16T11:10:02.493 INFO:teuthology.orchestra.run.smithi162.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-16T11:10:02.495 INFO:teuthology.orchestra.run.smithi162.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-16T11:10:02.826 INFO:teuthology.orchestra.run.smithi162.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-16T11:10:03.232 INFO:teuthology.orchestra.run.smithi162.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-16T11:10:03.233 INFO:teuthology.orchestra.run.smithi162.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-16T11:10:05.261 INFO:teuthology.orchestra.run.smithi165.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-16T11:10:05.266 INFO:teuthology.orchestra.run.smithi165.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-16T11:10:05.267 INFO:teuthology.orchestra.run.smithi165.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-16T11:10:06.472 INFO:teuthology.orchestra.run.smithi165.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-16T11:10:06.867 INFO:teuthology.orchestra.run.smithi162.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-16T11:10:06.897 INFO:teuthology.orchestra.run.smithi162.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-16T11:10:06.900 INFO:teuthology.orchestra.run.smithi162.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-16T11:10:07.127 INFO:teuthology.orchestra.run.smithi165.stdout:Fetched 201 MB in 18s (11.4 MB/s) 2024-09-16T11:10:07.307 INFO:teuthology.orchestra.run.smithi162.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-16T11:10:07.341 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-16T11:10:07.397 INFO:teuthology.orchestra.run.smithi165.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-16T11:10:07.402 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-16T11:10:07.515 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T11:10:07.886 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:07.942 INFO:teuthology.orchestra.run.smithi162.stdout:Fetched 201 MB in 18s (10.9 MB/s) 2024-09-16T11:10:08.044 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T11:10:08.112 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-16T11:10:08.167 INFO:teuthology.orchestra.run.smithi162.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-16T11:10:08.172 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-16T11:10:08.261 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T11:10:08.631 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:08.790 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T11:10:08.798 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:08.907 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T11:10:09.511 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:09.627 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-16T11:10:09.653 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T11:10:09.670 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:09.711 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:10.064 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rados. 2024-09-16T11:10:10.081 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:10.122 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:10.357 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T11:10:10.375 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:10.415 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:10.517 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T11:10:10.532 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:10.566 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:10.775 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rados. 2024-09-16T11:10:10.791 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:10.819 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T11:10:10.825 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:10.837 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:10.877 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:11.170 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T11:10:11.189 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:11.230 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T11:10:11.248 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:11.261 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:11.280 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:11.556 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T11:10:11.572 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:11.614 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:11.617 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T11:10:11.633 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-16T11:10:11.702 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-16T11:10:11.950 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T11:10:11.966 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:11.970 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T11:10:11.989 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:12.019 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:12.033 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:12.353 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T11:10:12.371 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-16T11:10:12.389 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-16T11:10:12.403 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-16T11:10:12.406 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-16T11:10:12.472 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T11:10:12.664 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T11:10:12.680 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:12.722 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:12.867 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-16T11:10:12.885 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-16T11:10:12.950 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T11:10:13.109 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-16T11:10:13.124 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-16T11:10:13.167 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T11:10:13.329 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T11:10:13.346 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-16T11:10:13.412 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T11:10:13.570 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-16T11:10:13.587 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-16T11:10:13.628 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T11:10:13.756 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T11:10:13.774 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-16T11:10:13.806 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T11:10:13.973 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T11:10:13.990 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-16T11:10:14.056 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T11:10:14.209 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T11:10:14.226 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-16T11:10:14.268 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T11:10:14.459 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T11:10:14.475 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-16T11:10:14.542 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T11:10:14.780 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T11:10:14.798 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T11:10:14.870 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T11:10:14.871 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:10:14.887 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-16T11:10:14.928 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T11:10:15.407 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T11:10:15.426 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T11:10:15.441 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T11:10:15.457 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T11:10:15.474 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:10:15.499 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:10:15.888 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T11:10:15.904 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T11:10:15.938 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:10:16.010 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T11:10:16.029 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T11:10:16.093 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:10:16.383 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-16T11:10:16.399 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-16T11:10:16.441 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T11:10:16.447 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T11:10:16.462 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-16T11:10:16.496 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:10:16.844 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package librgw2. 2024-09-16T11:10:16.860 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:16.911 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:16.924 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-16T11:10:16.942 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-16T11:10:16.983 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T11:10:17.427 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package librgw2. 2024-09-16T11:10:17.446 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:17.485 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:17.739 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T11:10:17.755 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:17.822 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:18.201 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liboath0. 2024-09-16T11:10:18.216 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-16T11:10:18.258 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-16T11:10:18.272 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T11:10:18.290 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:18.330 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:18.603 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T11:10:18.621 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:18.661 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:18.724 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liboath0. 2024-09-16T11:10:18.740 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-16T11:10:18.782 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-16T11:10:19.102 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T11:10:19.114 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-common. 2024-09-16T11:10:19.121 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:19.133 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:19.164 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:19.185 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:19.706 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-common. 2024-09-16T11:10:19.721 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:19.763 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:22.352 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-base. 2024-09-16T11:10:22.369 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:22.379 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-base. 2024-09-16T11:10:22.398 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:22.456 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:22.971 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:23.350 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cryptography. 2024-09-16T11:10:23.369 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-16T11:10:23.433 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T11:10:23.912 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T11:10:23.930 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-16T11:10:23.970 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-16T11:10:24.032 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cryptography. 2024-09-16T11:10:24.048 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-16T11:10:24.107 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T11:10:24.298 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-packaging. 2024-09-16T11:10:24.317 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-16T11:10:24.381 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-16T11:10:24.502 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T11:10:24.519 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-16T11:10:24.560 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-16T11:10:24.659 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T11:10:24.678 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:24.717 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-packaging. 2024-09-16T11:10:24.971 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-16T11:10:25.013 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-16T11:10:25.181 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T11:10:25.197 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-16T11:10:25.239 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T11:10:25.291 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T11:10:25.307 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:25.349 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:25.567 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T11:10:25.583 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-16T11:10:25.642 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-16T11:10:25.804 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T11:10:25.822 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-16T11:10:25.862 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T11:10:26.012 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T11:10:26.030 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-16T11:10:26.095 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-16T11:10:26.182 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T11:10:26.205 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-16T11:10:26.273 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-16T11:10:26.415 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jwt. 2024-09-16T11:10:26.434 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-16T11:10:26.473 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T11:10:26.644 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T11:10:26.660 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-16T11:10:26.701 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-16T11:10:26.843 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T11:10:26.859 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-16T11:10:26.901 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-16T11:10:27.063 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jwt. 2024-09-16T11:10:27.082 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-16T11:10:27.121 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T11:10:27.221 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T11:10:27.241 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-16T11:10:27.303 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-16T11:10:27.441 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T11:10:27.457 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-16T11:10:27.491 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-16T11:10:27.608 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-mako. 2024-09-16T11:10:27.623 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-16T11:10:27.657 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T11:10:27.869 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T11:10:27.889 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-16T11:10:27.927 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-16T11:10:28.044 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T11:10:28.062 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T11:10:28.102 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T11:10:28.274 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-mako. 2024-09-16T11:10:28.291 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-16T11:10:28.357 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T11:10:28.414 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T11:10:28.433 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-16T11:10:28.472 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T11:10:28.701 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T11:10:28.721 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T11:10:28.759 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T11:10:28.833 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-webob. 2024-09-16T11:10:28.850 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-16T11:10:28.891 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T11:10:29.122 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T11:10:29.139 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-16T11:10:29.180 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T11:10:29.268 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-16T11:10:29.284 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-16T11:10:29.326 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T11:10:29.517 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webob. 2024-09-16T11:10:29.534 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-16T11:10:29.600 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T11:10:29.639 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-bs4. 2024-09-16T11:10:29.656 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-16T11:10:29.689 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-16T11:10:29.953 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-16T11:10:29.968 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-16T11:10:30.011 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T11:10:30.067 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T11:10:30.084 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-16T11:10:30.128 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T11:10:30.381 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-bs4. 2024-09-16T11:10:30.399 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-16T11:10:30.439 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-16T11:10:30.520 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T11:10:30.536 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-16T11:10:30.578 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-16T11:10:30.867 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T11:10:30.881 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-paste. 2024-09-16T11:10:30.886 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-16T11:10:30.901 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-16T11:10:30.929 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T11:10:30.973 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T11:10:31.279 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T11:10:31.295 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-16T11:10:31.310 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T11:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-16T11:10:31.359 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-16T11:10:31.368 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T11:10:31.699 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-paste. 2024-09-16T11:10:31.715 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-16T11:10:31.746 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T11:10:31.757 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T11:10:31.766 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-16T11:10:31.804 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-16T11:10:32.104 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T11:10:32.122 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-16T11:10:32.181 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T11:10:32.183 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T11:10:32.201 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-16T11:10:32.241 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-16T11:10:32.534 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T11:10:32.551 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-16T11:10:32.581 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T11:10:32.592 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-16T11:10:32.599 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-16T11:10:32.673 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T11:10:32.971 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T11:10:32.987 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-16T11:10:33.029 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-16T11:10:33.053 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T11:10:33.069 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-16T11:10:33.111 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T11:10:33.381 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T11:10:33.397 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-16T11:10:33.473 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T11:10:33.506 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T11:10:33.523 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:33.564 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:33.853 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T11:10:33.869 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-16T11:10:33.911 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T11:10:33.967 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T11:10:33.983 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:34.025 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:34.323 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T11:10:34.340 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:34.381 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:34.537 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T11:10:34.552 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:34.621 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:34.802 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T11:10:34.819 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:34.860 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:35.364 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T11:10:35.380 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:35.421 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:35.450 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T11:10:35.466 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:35.508 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:36.293 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T11:10:36.311 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:36.351 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:37.719 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph. 2024-09-16T11:10:37.738 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:37.827 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:38.155 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T11:10:38.174 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:38.213 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:38.659 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T11:10:38.678 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:38.742 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:38.854 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph. 2024-09-16T11:10:38.871 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:38.912 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:39.265 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T11:10:39.285 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:39.295 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package cephadm. 2024-09-16T11:10:39.314 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:39.323 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:39.353 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:39.748 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T11:10:39.766 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-16T11:10:39.802 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T11:10:39.806 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T11:10:39.820 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:39.860 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:40.142 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T11:10:40.161 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:40.225 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:40.430 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package cephadm. 2024-09-16T11:10:40.447 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:40.488 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:40.620 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T11:10:40.639 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T11:10:40.703 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T11:10:40.841 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T11:10:40.857 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-16T11:10:40.891 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T11:10:40.965 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libjs-underscore. 2024-09-16T11:10:40.984 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-16T11:10:41.023 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T11:10:41.245 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T11:10:41.260 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:41.292 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-16T11:10:41.308 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-16T11:10:41.327 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:41.350 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T11:10:41.706 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T11:10:41.720 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-routes. 2024-09-16T11:10:41.725 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T11:10:41.738 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-16T11:10:41.764 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T11:10:41.778 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-16T11:10:42.075 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjs-underscore. 2024-09-16T11:10:42.092 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-16T11:10:42.133 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T11:10:42.178 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T11:10:42.198 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:42.248 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:42.445 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-16T11:10:42.463 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-16T11:10:42.504 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T11:10:42.907 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-routes. 2024-09-16T11:10:42.923 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-16T11:10:42.965 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-16T11:10:43.376 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T11:10:43.395 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:43.435 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:43.588 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-decorator. 2024-09-16T11:10:43.606 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-16T11:10:43.670 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T11:10:43.999 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-16T11:10:44.015 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-16T11:10:44.057 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T11:10:44.443 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-scipy. 2024-09-16T11:10:44.460 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-16T11:10:44.501 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-16T11:10:44.899 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-decorator. 2024-09-16T11:10:44.915 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-16T11:10:44.957 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T11:10:45.293 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-16T11:10:45.309 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-16T11:10:45.384 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T11:10:45.737 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-scipy. 2024-09-16T11:10:45.756 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-16T11:10:45.795 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-16T11:10:45.919 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-16T11:10:45.936 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-16T11:10:45.977 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T11:10:46.573 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T11:10:46.593 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-16T11:10:46.631 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-16T11:10:47.068 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T11:10:47.084 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-16T11:10:47.126 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T11:10:47.238 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-16T11:10:47.256 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-16T11:10:47.296 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T11:10:47.859 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T11:10:47.878 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-16T11:10:47.901 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T11:10:47.918 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:47.941 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-16T11:10:47.983 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:48.329 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T11:10:48.345 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-16T11:10:48.412 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T11:10:49.146 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T11:10:49.163 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-16T11:10:49.204 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-16T11:10:49.229 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T11:10:49.247 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:49.287 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:49.565 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T11:10:49.581 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-16T11:10:49.623 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T11:10:50.027 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T11:10:50.044 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-16T11:10:50.085 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T11:10:50.389 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T11:10:50.407 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-16T11:10:50.439 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-16T11:10:50.450 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T11:10:50.466 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-16T11:10:50.507 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-16T11:10:50.809 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T11:10:50.825 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-16T11:10:50.867 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-16T11:10:50.877 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T11:10:50.895 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-16T11:10:50.935 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T11:10:51.230 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-blinker. 2024-09-16T11:10:51.250 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-16T11:10:51.295 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T11:10:51.322 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T11:10:51.342 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-16T11:10:51.380 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T11:10:51.631 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-16T11:10:51.648 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-16T11:10:51.690 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T11:10:51.700 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T11:10:51.716 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-16T11:10:51.758 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-16T11:10:52.085 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T11:10:52.101 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-16T11:10:52.121 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T11:10:52.137 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-16T11:10:52.143 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T11:10:52.179 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-16T11:10:52.532 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-blinker. 2024-09-16T11:10:52.538 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T11:10:52.551 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-16T11:10:52.555 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-16T11:10:52.596 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T11:10:52.615 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T11:10:52.927 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-16T11:10:52.943 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-16T11:10:52.958 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T11:10:52.974 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-16T11:10:52.985 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T11:10:53.018 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T11:10:53.356 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T11:10:53.375 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-16T11:10:53.425 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T11:10:53.713 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T11:10:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:53.762 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T11:10:53.779 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-16T11:10:53.820 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T11:10:54.106 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T11:10:54.123 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-16T11:10:54.197 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-16T11:10:54.207 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T11:10:54.224 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-16T11:10:54.257 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T11:10:54.584 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T11:10:54.600 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T11:10:54.675 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:10:54.962 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package jq. 2024-09-16T11:10:54.979 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T11:10:54.998 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T11:10:55.016 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:10:55.020 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:10:55.056 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:55.298 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package socat. 2024-09-16T11:10:55.314 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-16T11:10:55.356 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-16T11:10:55.443 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T11:10:55.462 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-16T11:10:55.501 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-16T11:10:55.729 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T11:10:55.746 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-16T11:10:55.787 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-16T11:10:55.876 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T11:10:55.892 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T11:10:55.959 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:10:56.184 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-test. 2024-09-16T11:10:56.200 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:56.242 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:10:56.262 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package jq. 2024-09-16T11:10:56.280 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-16T11:10:56.320 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:10:56.608 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package socat. 2024-09-16T11:10:56.624 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-16T11:10:56.691 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-16T11:10:57.055 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T11:10:57.072 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-16T11:10:57.113 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-16T11:10:57.510 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-test. 2024-09-16T11:10:57.527 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:10:57.559 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:00.930 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T11:11:00.947 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:11:00.988 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:01.335 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-16T11:11:01.351 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T11:11:01.409 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:01.971 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T11:11:01.988 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:11:02.029 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:02.297 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T11:11:02.313 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-16T11:11:02.355 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:02.441 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T11:11:02.459 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-16T11:11:02.517 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T11:11:02.727 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-16T11:11:02.746 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T11:11:02.818 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:02.903 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-16T11:11:02.923 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T11:11:02.961 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:03.372 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T11:11:03.391 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:11:03.430 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:03.902 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T11:11:03.920 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-16T11:11:03.978 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T11:11:04.375 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-16T11:11:04.392 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-16T11:11:04.424 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:04.776 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-16T11:11:04.796 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-16T11:11:04.834 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-16T11:11:05.195 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-16T11:11:05.213 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-16T11:11:05.278 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T11:11:05.581 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-attr. 2024-09-16T11:11:05.602 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-16T11:11:05.631 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-16T11:11:05.942 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-webencodings. 2024-09-16T11:11:05.960 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-16T11:11:06.016 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T11:11:06.190 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-16T11:11:06.212 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-16T11:11:06.282 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-16T11:11:06.353 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-html5lib. 2024-09-16T11:11:06.369 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-16T11:11:06.411 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-16T11:11:06.651 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-16T11:11:06.669 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-16T11:11:06.709 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T11:11:06.789 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-16T11:11:06.805 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-16T11:11:06.847 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T11:11:07.071 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-attr. 2024-09-16T11:11:07.091 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-16T11:11:07.129 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-16T11:11:07.342 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T11:11:07.360 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T11:11:07.399 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T11:11:07.474 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-webencodings. 2024-09-16T11:11:07.492 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-16T11:11:07.553 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T11:11:07.761 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T11:11:07.778 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-16T11:11:07.820 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-16T11:11:07.885 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-html5lib. 2024-09-16T11:11:07.901 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-16T11:11:07.943 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-16T11:11:08.338 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-16T11:11:08.359 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-16T11:11:08.404 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T11:11:08.606 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T11:11:08.624 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-16T11:11:08.663 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T11:11:08.933 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T11:11:08.952 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T11:11:08.991 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T11:11:09.050 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-py. 2024-09-16T11:11:09.068 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-16T11:11:09.108 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T11:11:09.337 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T11:11:09.354 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-16T11:11:09.503 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T11:11:09.511 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-16T11:11:09.520 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-16T11:11:09.552 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T11:11:09.897 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T11:11:09.914 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-16T11:11:09.980 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T11:11:10.206 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T11:11:10.225 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-16T11:11:10.264 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T11:11:10.308 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T11:11:10.327 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-16T11:11:10.366 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-16T11:11:10.618 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-py. 2024-09-16T11:11:10.636 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-16T11:11:10.701 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T11:11:10.763 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T11:11:10.783 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-16T11:11:10.821 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T11:11:11.041 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T11:11:11.060 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-16T11:11:11.099 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T11:11:11.149 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T11:11:11.169 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-16T11:11:11.232 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T11:11:11.477 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T11:11:11.495 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-16T11:11:11.543 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T11:11:11.871 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T11:11:11.888 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-16T11:11:11.954 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-16T11:11:12.003 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package radosgw. 2024-09-16T11:11:12.022 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:11:12.094 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:12.316 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T11:11:12.333 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-16T11:11:12.374 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T11:11:12.717 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T11:11:12.748 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-16T11:11:12.785 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T11:11:13.642 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package radosgw. 2024-09-16T11:11:13.663 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:11:13.700 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:13.764 INFO:teuthology.orchestra.run.smithi165.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T11:11:13.781 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:11:13.847 INFO:teuthology.orchestra.run.smithi165.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:14.187 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-16T11:11:14.504 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T11:11:14.663 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T11:11:14.822 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T11:11:15.015 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:15.537 INFO:teuthology.orchestra.run.smithi162.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T11:11:15.559 INFO:teuthology.orchestra.run.smithi162.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-16T11:11:15.596 INFO:teuthology.orchestra.run.smithi162.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:15.651 INFO:teuthology.orchestra.run.smithi165.stdout:Adding system user cephadm....done 2024-09-16T11:11:15.855 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:15.924 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-16T11:11:15.981 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T11:11:16.260 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-16T11:11:16.366 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T11:11:16.412 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-16T11:11:16.579 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-16T11:11:16.667 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:16.772 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:16.802 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-16T11:11:16.902 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T11:11:17.204 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-16T11:11:17.426 INFO:teuthology.orchestra.run.smithi162.stdout:Adding system user cephadm....done 2024-09-16T11:11:17.621 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:17.656 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T11:11:17.747 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-16T11:11:18.007 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-16T11:11:18.140 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T11:11:18.300 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T11:11:18.433 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-16T11:11:18.568 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-16T11:11:18.584 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T11:11:18.719 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-16T11:11:18.861 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:19.029 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-16T11:11:19.190 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T11:11:19.312 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-16T11:11:19.473 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-16T11:11:19.597 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T11:11:19.774 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-16T11:11:19.956 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T11:11:20.067 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-16T11:11:20.082 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T11:11:20.376 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-16T11:11:20.483 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-16T11:11:20.728 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:20.818 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T11:11:21.004 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-16T11:11:21.094 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T11:11:21.129 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-16T11:11:21.462 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-16T11:11:21.462 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T11:11:21.729 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T11:11:21.765 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-16T11:11:21.891 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-16T11:11:22.072 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-16T11:11:22.293 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-16T11:11:22.398 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-16T11:11:22.585 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-16T11:11:22.691 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T11:11:22.842 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T11:11:22.853 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-16T11:11:22.976 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T11:11:23.205 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-16T11:11:23.236 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T11:11:23.539 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-16T11:11:23.662 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T11:11:23.873 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-16T11:11:24.014 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T11:11:24.183 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-16T11:11:24.415 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-16T11:11:24.524 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-16T11:11:24.575 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:24.660 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-16T11:11:24.803 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-16T11:11:24.884 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:11:25.011 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-16T11:11:25.079 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-16T11:11:25.379 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T11:11:25.464 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-16T11:11:25.496 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-16T11:11:25.647 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-16T11:11:25.866 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-16T11:11:25.932 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T11:11:26.226 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-16T11:11:26.291 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:11:26.377 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:26.402 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T11:11:26.537 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T11:11:26.679 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:11:26.846 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-16T11:11:26.855 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T11:11:27.139 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-16T11:11:27.189 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-16T11:11:27.357 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-16T11:11:27.482 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T11:11:27.483 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-16T11:11:27.616 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T11:11:27.793 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-16T11:11:27.917 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-16T11:11:28.152 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:11:28.260 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T11:11:28.262 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-16T11:11:28.421 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-16T11:11:28.544 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T11:11:28.748 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-16T11:11:28.866 INFO:teuthology.orchestra.run.smithi165.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-16T11:11:29.049 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-16T11:11:29.175 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-16T11:11:29.306 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T11:11:29.366 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-16T11:11:29.518 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T11:11:29.615 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-16T11:11:29.794 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-16T11:11:30.008 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T11:11:30.062 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-16T11:11:30.168 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T11:11:30.277 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T11:11:30.338 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-16T11:11:30.587 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T11:11:30.748 INFO:teuthology.orchestra.run.smithi162.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-16T11:11:30.904 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:31.024 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-16T11:11:31.030 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-16T11:11:31.150 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-16T11:11:31.501 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-16T11:11:31.861 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-16T11:11:32.004 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-16T11:11:32.130 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-16T11:11:32.511 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:32.523 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-16T11:11:32.645 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:11:32.754 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-16T11:11:32.807 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:32.942 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-16T11:11:33.156 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-16T11:11:33.457 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-16T11:11:33.808 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:11:33.934 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-16T11:11:34.319 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:34.419 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T11:11:34.422 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:34.582 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:11:34.708 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-16T11:11:34.729 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:34.881 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T11:11:35.110 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-16T11:11:35.206 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T11:11:35.419 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-16T11:11:35.704 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-16T11:11:35.855 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-16T11:11:36.198 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:36.295 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:36.357 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-16T11:11:36.396 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T11:11:36.555 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-16T11:11:36.681 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:11:36.692 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:36.832 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-16T11:11:36.843 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-16T11:11:37.133 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T11:11:37.186 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-16T11:11:37.284 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-16T11:11:37.585 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:37.719 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:37.931 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T11:11:37.931 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T11:11:38.209 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:38.360 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-16T11:11:38.421 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:38.494 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-16T11:11:38.645 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-16T11:11:38.716 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T11:11:38.772 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-16T11:11:39.120 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-16T11:11:39.248 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-16T11:11:39.481 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T11:11:39.558 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:39.717 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:39.947 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T11:11:39.947 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T11:11:40.153 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T11:11:40.522 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:40.673 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:40.744 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:40.824 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:40.950 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:40.997 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-16T11:11:41.117 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-16T11:11:41.282 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T11:11:41.419 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-16T11:11:41.627 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-16T11:11:41.721 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:41.871 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:41.997 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T11:11:42.021 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:42.147 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:42.307 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:42.433 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-16T11:11:42.894 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-16T11:11:42.933 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:43.261 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:43.388 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:43.555 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-16T11:11:43.561 INFO:teuthology.orchestra.run.smithi165.stdout:Adding group ceph....done 2024-09-16T11:11:43.992 INFO:teuthology.orchestra.run.smithi165.stdout:Adding system user ceph....done 2024-09-16T11:11:44.163 INFO:teuthology.orchestra.run.smithi165.stdout:Setting system user ceph properties....done 2024-09-16T11:11:44.181 INFO:teuthology.orchestra.run.smithi165.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T11:11:44.335 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T11:11:44.492 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:44.671 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T11:11:45.111 INFO:teuthology.orchestra.run.smithi162.stdout:Adding group ceph....done 2024-09-16T11:11:45.117 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:45.276 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:45.521 INFO:teuthology.orchestra.run.smithi162.stdout:Adding system user ceph....done 2024-09-16T11:11:45.556 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:11:45.556 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:11:45.722 INFO:teuthology.orchestra.run.smithi162.stdout:Setting system user ceph properties....done 2024-09-16T11:11:45.744 INFO:teuthology.orchestra.run.smithi162.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T11:11:45.890 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T11:11:46.188 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T11:11:46.228 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:46.603 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T11:11:46.668 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:46.794 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:47.038 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:47.090 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:11:47.090 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T11:11:47.273 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T11:11:47.274 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T11:11:47.706 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:47.713 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:47.939 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T11:11:47.939 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T11:11:48.116 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T11:11:48.407 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:48.556 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:48.821 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T11:11:48.821 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T11:11:48.912 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T11:11:48.912 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T11:11:49.249 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:49.367 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:49.478 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T11:11:49.478 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T11:11:49.493 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:49.661 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:49.885 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T11:11:49.885 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T11:11:49.942 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:50.371 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:50.419 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T11:11:50.419 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T11:11:50.514 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:50.656 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:50.791 INFO:teuthology.orchestra.run.smithi165.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:50.827 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:50.951 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-16T11:11:51.144 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-16T11:11:51.195 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:51.445 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-16T11:11:51.664 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:51.913 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T11:11:51.913 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T11:11:52.348 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:52.518 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:52.659 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:52.818 INFO:teuthology.orchestra.run.smithi162.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-16T11:11:52.945 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-16T11:11:53.113 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-16T11:11:53.398 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-16T11:11:54.289 INFO:teuthology.orchestra.run.smithi165.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-16T11:11:54.945 INFO:teuthology.orchestra.run.smithi165.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T11:11:54.950 DEBUG:teuthology.parallel:result is None 2024-09-16T11:11:56.192 INFO:teuthology.orchestra.run.smithi162.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-16T11:11:56.855 INFO:teuthology.orchestra.run.smithi162.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T11:11:56.861 DEBUG:teuthology.parallel:result is None 2024-09-16T11:11:56.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:11:57.019 DEBUG:teuthology.orchestra.run.smithi162:> dpkg-query -W -f '${Version}' ceph 2024-09-16T11:11:57.042 INFO:teuthology.orchestra.run.smithi162.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-16T11:11:57.043 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-16T11:11:57.043 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-16T11:11:57.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:11:57.195 DEBUG:teuthology.orchestra.run.smithi165:> dpkg-query -W -f '${Version}' ceph 2024-09-16T11:11:57.218 INFO:teuthology.orchestra.run.smithi165.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-16T11:11:57.218 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-16T11:11:57.218 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-16T11:11:57.220 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T11:11:57.221 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:57.221 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T11:11:57.238 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:57.238 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T11:11:57.276 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T11:11:57.276 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:57.276 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T11:11:57.290 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T11:11:57.347 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:57.348 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T11:11:57.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T11:11:57.373 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T11:11:57.373 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:57.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T11:11:57.387 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T11:11:57.443 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:57.443 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T11:11:57.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T11:11:57.469 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T11:11:57.470 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:57.470 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T11:11:57.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T11:11:57.541 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:57.541 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T11:11:57.557 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T11:11:57.619 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T11:11:57.698 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T11:11:57.699 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /var/log/ceph 2024-09-16T11:11:57.701 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2024-09-16T11:11:57.717 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T11:11:57.717 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T11:11:57.758 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T11:11:57.773 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T11:11:57.773 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T11:11:57.775 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T11:11:57.844 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T11:11:57.845 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T11:11:57.845 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_copy_on_read}', 'email': 'sangadi@redhat.com', 'first_in_suite': False, 'job_id': '7907154', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_sangadi@teuthology', 'priority': 130, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9764, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi162.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_10:44:09', 'tube': 'smithi', 'user': 'sangadi', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T11:11:57.845 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T11:11:57.848 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T11:11:57.855 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T11:11:57.857 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T11:11:57.873 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:57.873 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T11:11:57.921 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T11:11:57.922 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-09-16T11:11:57.928 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T11:11:57.928 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:57.928 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-16T11:11:57.929 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:57.929 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 11:08:11.124323310 +0000 2024-09-16T11:11:57.929 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 11:08:10.932327738 +0000 2024-09-16T11:11:57.929 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 11:08:10.932327738 +0000 2024-09-16T11:11:57.929 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-09-16T11:11:57.929 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T11:11:57.942 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T11:11:57.942 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T11:11:57.942 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000425357 s, 1.2 MB/s 2024-09-16T11:11:57.944 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T11:11:57.993 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-09-16T11:11:57.999 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 11:08:11.544313622 +0000 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 11:08:11.332318514 +0000 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 11:08:11.332318514 +0000 2024-09-16T11:11:58.000 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-09-16T11:11:58.000 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T11:11:58.015 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T11:11:58.015 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T11:11:58.015 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000376805 s, 1.4 MB/s 2024-09-16T11:11:58.016 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T11:11:58.023 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-09-16T11:11:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T11:11:58.071 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.071 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-16T11:11:58.071 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.071 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 11:08:11.992303287 +0000 2024-09-16T11:11:58.071 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 11:08:11.772308362 +0000 2024-09-16T11:11:58.072 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 11:08:11.772308362 +0000 2024-09-16T11:11:58.072 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-09-16T11:11:58.072 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T11:11:58.084 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T11:11:58.084 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T11:11:58.084 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000415361 s, 1.2 MB/s 2024-09-16T11:11:58.085 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T11:11:58.094 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-16 11:08:12.436293044 +0000 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-16 11:08:12.236297659 +0000 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-16 11:08:12.236297659 +0000 2024-09-16T11:11:58.101 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: - 2024-09-16T11:11:58.101 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T11:11:58.157 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-16T11:11:58.157 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-16T11:11:58.157 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.00034001 s, 1.5 MB/s 2024-09-16T11:11:58.158 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T11:11:58.167 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:11:58.167 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:58.167 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T11:11:58.174 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T11:11:58.174 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-16 11:08:08.150362304 +0000 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-16 11:08:07.962338599 +0000 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-16 11:08:07.962338599 +0000 2024-09-16T11:11:58.182 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-09-16T11:11:58.183 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T11:11:58.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-16T11:11:58.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-16T11:11:58.198 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000426297 s, 1.2 MB/s 2024-09-16T11:11:58.199 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T11:11:58.206 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-16 11:08:08.570414779 +0000 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-16 11:08:08.378390790 +0000 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-16 11:08:08.378390790 +0000 2024-09-16T11:11:58.213 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-09-16T11:11:58.213 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T11:11:58.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-16T11:11:58.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-16T11:11:58.228 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000318055 s, 1.6 MB/s 2024-09-16T11:11:58.229 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T11:11:58.238 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-16 11:08:09.010469727 +0000 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-16 11:08:08.818445763 +0000 2024-09-16T11:11:58.244 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-16 11:08:08.818445763 +0000 2024-09-16T11:11:58.245 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-09-16T11:11:58.245 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T11:11:58.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-16T11:11:58.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-16T11:11:58.303 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000421634 s, 1.2 MB/s 2024-09-16T11:11:58.304 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T11:11:58.311 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-09-16T11:11:58.316 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T11:11:58.316 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:11:58.316 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-16T11:11:58.316 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:11:58.316 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-09-16 11:08:09.462523938 +0000 2024-09-16T11:11:58.316 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-09-16 11:08:09.250498512 +0000 2024-09-16T11:11:58.317 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-09-16 11:08:09.250498512 +0000 2024-09-16T11:11:58.317 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: - 2024-09-16T11:11:58.317 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T11:11:58.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-09-16T11:11:58.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-09-16T11:11:58.375 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000364736 s, 1.4 MB/s 2024-09-16T11:11:58.376 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T11:11:58.383 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:11:58.383 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T11:11:58.383 INFO:tasks.ceph:Generating config... 2024-09-16T11:11:58.384 INFO:tasks.ceph:[client] rbd cache = True 2024-09-16T11:11:58.385 INFO:tasks.ceph:[client] rbd clone copy on read = True 2024-09-16T11:11:58.385 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T11:11:58.385 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T11:11:58.385 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-16T11:11:58.385 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T11:11:58.385 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T11:11:58.385 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T11:11:58.385 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T11:11:58.385 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T11:11:58.385 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T11:11:58.385 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T11:11:58.385 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-16T11:11:58.385 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T11:11:58.385 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T11:11:58.385 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T11:11:58.386 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T11:11:58.386 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T11:11:58.387 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T11:11:58.421 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T11:11:58.423 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T11:11:58.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:11:58.517 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.162'), ('mon.c', '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]'), ('mon.b', '172.21.15.165')] 2024-09-16T11:11:58.517 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],172.21.15.165', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd clone copy on read': True}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-16T11:11:58.518 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:58.518 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T11:11:58.525 DEBUG:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.162 --addv c '[v2:172.21.15.162:3301,v1:172.21.15.162:6790]' --add b 172.21.15.165 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:11:58.562 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:11:58.562 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool: generated fsid 7e3ff0c6-8090-4b11-95d6-fd611813de49 2024-09-16T11:11:58.562 INFO:teuthology.orchestra.run.smithi162.stdout:setting min_mon_release = octopus 2024-09-16T11:11:58.562 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 0 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:fsid 7e3ff0c6-8090-4b11-95d6-fd611813de49 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:last_changed 2024-09-16T11:11:58.562033+0000 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-09-16T11:11:58.562033+0000 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:min_mon_release 15 (octopus) 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:election_strategy: 1 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:0: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.a 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.b 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:2: [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] mon.c 2024-09-16T11:11:58.563 INFO:teuthology.orchestra.run.smithi162.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T11:11:58.564 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T11:11:58.569 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7e3ff0c6-8090-4b11-95d6-fd611813de49... 2024-09-16T11:11:58.570 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T11:11:58.614 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T11:11:58.648 INFO:teuthology.orchestra.run.smithi162.stdout:[global] 2024-09-16T11:11:58.648 INFO:teuthology.orchestra.run.smithi162.stdout: chdir = "" 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: auth supported = cephx 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: filestore xattr use omap = true 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: mon clock drift allowed = 1.000 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: osd crush chooseleaf type = 0 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: auth debug = true 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on old message = true 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: ms die on bug = true 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.649 INFO:teuthology.orchestra.run.smithi162.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon pg warn max object skew = 0 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default size = 2 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow primary affinity = true 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd allow pg remap = true 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on legacy crush tunables = false 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on crush straw calc version zero = false 2024-09-16T11:11:58.650 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on no sortbitwise = false 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on osd down out interval zero = false 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on too few osds = false 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon_allow_pool_size_one = true 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: osd default data pool replay window = 5 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon allow pool delete = true 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.651 INFO:teuthology.orchestra.run.smithi162.stdout: mon cluster log file level = debug 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: debug asserts on shutdown = true 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: mon health detail to clog = false 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: mon host = "172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],172.21.15.165" 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: mon client directed command retry = 5 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: mon warn on pool no app = False 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: ms inject socket failures = 5000 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: fsid = 7e3ff0c6-8090-4b11-95d6-fd611813de49 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout:[osd] 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: osd journal size = 100 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub load threshold = 5.0 2024-09-16T11:11:58.652 INFO:teuthology.orchestra.run.smithi162.stdout: osd scrub max interval = 600 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd mclock profile = high_recovery_ops 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd recover clone overlap = true 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd recovery max chunk = 1048576 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug shutdown = true 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug op order = true 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug verify stray on activate = true 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd open classes on start = true 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug pg log writeout = true 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: osd deep scrub update digest min age = 30 2024-09-16T11:11:58.653 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: osd map max advance = 10 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: journal zero on create = true 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: filestore ondisk finisher threads = 3 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: filestore apply finisher threads = 3 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: bdev debug aio = true 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: osd debug misdirected ops = true 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: bdev async discard = True 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: bdev enable discard = True 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore allocator = bitmap 2024-09-16T11:11:58.654 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore block size = 96636764160 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: bluestore fsck on mount = True 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluefs = 1/20 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: debug bluestore = 1/20 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: debug osd = 20 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: debug rocksdb = 4/10 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd full ratio = 0.9 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd nearfull ratio = 0.8 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: osd failsafe full ratio = 0.95 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: osd max markdown count = 1000 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: osd objectstore = bluestore 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: osd shutdown pgref assert = True 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.655 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr] 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug mgr = 20 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: mgr/telemetry/nag = false 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout:[mon] 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug ms = 1 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug mon = 20 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug paxos = 20 2024-09-16T11:11:58.656 INFO:teuthology.orchestra.run.smithi162.stdout: debug auth = 20 2024-09-16T11:11:58.657 INFO:teuthology.orchestra.run.smithi162.stdout: mon data avail warn = 5 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: mon mgr mkfs grace = 240 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd reporter subtree level = osd 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: mon osd prime pg temp = true 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T11:11:58.660 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: # 1m isn't quite enough 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: mon_down_mkfs_grace = 2m 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: mon_warn_on_filestore_osds = false 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout:[client] 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: rgw cache enabled = true 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable ops log = true 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: rgw enable usage log = true 2024-09-16T11:11:58.661 INFO:teuthology.orchestra.run.smithi162.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T11:11:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T11:11:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: rbd cache = True 2024-09-16T11:11:58.662 INFO:teuthology.orchestra.run.smithi162.stdout: rbd clone copy on read = True 2024-09-16T11:11:58.662 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.a] 2024-09-16T11:11:58.662 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.c] 2024-09-16T11:11:58.662 INFO:teuthology.orchestra.run.smithi162.stdout:[mon.b] 2024-09-16T11:11:58.663 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.664 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.665 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-09-16T11:11:58.666 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],172.21.15.165" 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: mon client directed command retry = 5 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on pool no app = False 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: ms inject socket failures = 5000 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 7e3ff0c6-8090-4b11-95d6-fd611813de49 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-09-16T11:11:58.667 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.668 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.669 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: bdev async discard = True 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: bdev enable discard = True 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore allocator = bitmap 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 1/20 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 1/20 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 4/10 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-09-16T11:11:58.670 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: osd max markdown count = 1000 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: osd shutdown pgref assert = True 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:11:58.671 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-09-16T11:11:58.672 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.672 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-09-16T11:11:58.672 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-16T11:11:58.672 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T11:11:58.673 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T11:11:58.674 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T11:11:58.674 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.674 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T11:11:58.674 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T11:11:58.674 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T11:11:58.675 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T11:11:58.676 INFO:teuthology.orchestra.run.smithi165.stdout: rbd cache = True 2024-09-16T11:11:58.676 INFO:teuthology.orchestra.run.smithi165.stdout: rbd clone copy on read = True 2024-09-16T11:11:58.676 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-09-16T11:11:58.676 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-09-16T11:11:58.676 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-09-16T11:11:58.676 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T11:11:58.677 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T11:11:58.725 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T11:11:58.726 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:58.726 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T11:11:58.733 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:58.733 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T11:11:58.738 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:11:58.738 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:58.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T11:11:58.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:11:58.758 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:11:58.758 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:11:58.764 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:11:58.764 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:58.764 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T11:11:58.764 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:11:58.792 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:11:58.792 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:11:58.799 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T11:11:58.799 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T11:11:58.799 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T11:11:58.843 INFO:teuthology.orchestra.run.smithi162.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T11:11:58.846 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T11:11:58.894 INFO:teuthology.orchestra.run.smithi165.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T11:11:58.896 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T11:11:58.897 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T11:11:58.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T11:11:58.933 INFO:teuthology.orchestra.run.smithi162.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T11:11:58.945 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T11:11:58.981 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T11:11:58.994 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T11:11:58.995 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi162.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T11:11:58.995 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T11:11:59.009 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:11:59.009 INFO:tasks.ceph:role: osd.0 2024-09-16T11:11:59.010 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:11:59.010 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:11:59.557 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:11:59.558 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:11:59.558 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:11:59.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T11:11:59.559 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T11:11:59.664 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T11:11:59.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:11:59.680 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:11:59.681 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T11:11:59.740 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:11:59.741 INFO:tasks.ceph:role: osd.1 2024-09-16T11:11:59.741 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:11:59.741 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T11:12:00.251 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:00.251 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:00.252 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:00.254 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T11:12:00.254 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T11:12:00.292 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T11:12:00.344 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:00.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:00.345 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T11:12:00.358 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:12:00.358 INFO:tasks.ceph:role: osd.2 2024-09-16T11:12:00.358 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:12:00.358 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T11:12:00.840 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:00.840 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:00.840 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:00.841 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:00.842 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T11:12:00.843 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T11:12:00.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T11:12:00.925 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:00.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:00.927 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T11:12:00.937 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T11:12:00.937 INFO:tasks.ceph:role: osd.3 2024-09-16T11:12:00.937 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:12:00.937 DEBUG:teuthology.orchestra.run.smithi162:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: = reflink=1 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:01.423 INFO:teuthology.orchestra.run.smithi162.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:01.423 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi162.front.sepia.ceph.com -o noatime 2024-09-16T11:12:01.424 DEBUG:teuthology.orchestra.run.smithi162:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T11:12:01.475 DEBUG:teuthology.orchestra.run.smithi162:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T11:12:01.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:01.487 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:01.487 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:01.557 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:01.555+0000 7fd5b514f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T11:12:01.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:01.555+0000 7fd5b514f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T11:12:01.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:01.555+0000 7fd5b514f540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:12:01.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:01.555+0000 7fd5b514f540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:12:01.558 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:01.555+0000 7fd5b514f540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:12:01.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:01.555+0000 7fd5b514f540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T11:12:02.133 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:02.131+0000 7fd5b514f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:02.920 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:02.919+0000 7fd5b514f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:03.710 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:03.707+0000 7fd5b514f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:04.573 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T11:12:04.586 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:04.672 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:04.667+0000 7fa4a6e0d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T11:12:04.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:04.667+0000 7fa4a6e0d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T11:12:04.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:04.667+0000 7fa4a6e0d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:12:04.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:04.671+0000 7fa4a6e0d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:12:04.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:04.671+0000 7fa4a6e0d540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:12:04.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:04.671+0000 7fa4a6e0d540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T11:12:05.230 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:05.227+0000 7fa4a6e0d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:06.027 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:06.023+0000 7fa4a6e0d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:06.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:06.811+0000 7fa4a6e0d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:07.664 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T11:12:07.678 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:07.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:07.711+0000 7f8861ef2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T11:12:07.716 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:07.711+0000 7f8861ef2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T11:12:07.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:07.711+0000 7f8861ef2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:12:07.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:07.711+0000 7f8861ef2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:12:07.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:07.711+0000 7f8861ef2540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:12:07.717 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:07.715+0000 7f8861ef2540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T11:12:08.271 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:08.267+0000 7f8861ef2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:09.063 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:09.059+0000 7f8861ef2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:09.850 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:09.847+0000 7f8861ef2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:10.708 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T11:12:10.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:10.807+0000 7f3692a08540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T11:12:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:10.807+0000 7f3692a08540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T11:12:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:10.807+0000 7f3692a08540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T11:12:10.809 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:10.807+0000 7f3692a08540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T11:12:10.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:10.807+0000 7f3692a08540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T11:12:10.810 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:10.807+0000 7f3692a08540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T11:12:11.373 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:11.371+0000 7f3692a08540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:12.163 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:12.159+0000 7f3692a08540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:12.940 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:12:12.939+0000 7f3692a08540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:13.792 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T11:12:13.806 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T11:12:13.825 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:12:13.825 INFO:tasks.ceph:role: osd.4 2024-09-16T11:12:13.825 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:12:13.826 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T11:12:14.361 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:14.361 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:14.361 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:14.362 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:14.363 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-09-16T11:12:14.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T11:12:14.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T11:12:14.491 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:14.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:14.492 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T11:12:14.502 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:12:14.502 INFO:tasks.ceph:role: osd.5 2024-09-16T11:12:14.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:12:14.502 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:15.000 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:15.001 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:15.001 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:15.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-09-16T11:12:15.002 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T11:12:15.052 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T11:12:15.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:15.064 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:15.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T11:12:15.124 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:12:15.125 INFO:tasks.ceph:role: osd.6 2024-09-16T11:12:15.125 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:12:15.125 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T11:12:15.623 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:15.624 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:15.626 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-09-16T11:12:15.626 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T11:12:15.688 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T11:12:15.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:15.702 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:15.702 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T11:12:15.760 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T11:12:15.760 INFO:tasks.ceph:role: osd.7 2024-09-16T11:12:15.760 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:12:15.761 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T11:12:16.264 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:12:16.265 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:12:16.267 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-09-16T11:12:16.267 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T11:12:16.335 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T11:12:16.348 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T11:12:16.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:12:16.349 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:16.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:16.609+0000 7f15aa50d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T11:12:16.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:16.609+0000 7f15aa50d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T11:12:16.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:16.609+0000 7f15aa50d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T11:12:16.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:16.609+0000 7f15aa50d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T11:12:16.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:16.609+0000 7f15aa50d540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T11:12:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:16.617+0000 7f15aa50d540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T11:12:17.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:17.177+0000 7f15aa50d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:17.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:17.965+0000 7f15aa50d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:18.748+0000 7f15aa50d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:19.601 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T11:12:19.615 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:19.644+0000 7f8840e54540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T11:12:19.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:19.644+0000 7f8840e54540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T11:12:19.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:19.644+0000 7f8840e54540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T11:12:19.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:19.644+0000 7f8840e54540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T11:12:19.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:19.644+0000 7f8840e54540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T11:12:19.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:19.644+0000 7f8840e54540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T11:12:20.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:20.200+0000 7f8840e54540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:20.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:20.988+0000 7f8840e54540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:21.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:21.780+0000 7f8840e54540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:22.648 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T11:12:22.708 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:22.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:22.736+0000 7f7004fd0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T11:12:22.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:22.736+0000 7f7004fd0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T11:12:22.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:22.736+0000 7f7004fd0540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T11:12:22.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:22.736+0000 7f7004fd0540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T11:12:22.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:22.736+0000 7f7004fd0540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T11:12:22.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:22.740+0000 7f7004fd0540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T11:12:23.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:23.292+0000 7f7004fd0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:24.088+0000 7f7004fd0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:24.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:24.880+0000 7f7004fd0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:25.728 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T11:12:25.744 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:25.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:25.816+0000 7f8a6cd20540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T11:12:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:25.816+0000 7f8a6cd20540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T11:12:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:25.820+0000 7f8a6cd20540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T11:12:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:25.820+0000 7f8a6cd20540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T11:12:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:25.820+0000 7f8a6cd20540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T11:12:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:25.820+0000 7f8a6cd20540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T11:12:26.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:26.384+0000 7f8a6cd20540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:27.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:27.164+0000 7f8a6cd20540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:27.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:12:27.952+0000 7f8a6cd20540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:12:28.799 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T11:12:28.813 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T11:12:28.813 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:28.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T11:12:28.830 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:28.830 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T11:12:28.887 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:28.888 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T11:12:28.897 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:28.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T11:12:28.910 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:28.911 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T11:12:28.968 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:28.968 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T11:12:28.984 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:28.984 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T11:12:29.039 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:29.039 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T11:12:29.049 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:29.049 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T11:12:29.059 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:29.059 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T11:12:29.115 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:29.116 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T11:12:29.122 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:29.123 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T11:12:29.128 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T11:12:29.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T11:12:29.130 DEBUG:teuthology.orchestra.run.smithi165:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.y] 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQB+EuhmaAJBMhAAaCyw39s/Ehxo0JmIh2ICgQ== 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.0] 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCBEuhmI6IwIRAAW3jAroOUVxUiSrBRFTPHag== 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.1] 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCEEuhmTqkLKBAAl+3j8/Cy2roZM7JKMkjLsw== 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.2] 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCHEuhmxx2dKhAAc5ZeuxN2PkyjewfHdvDUrg== 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.3] 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCKEuhmH3YwMBAA13gnux6DEJiJiRR/D8+i1g== 2024-09-16T11:12:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:[mgr.x] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQB+EuhmzkIXNRAA5U/JfRsInu3sdQbPULjbxA== 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.4] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCQEuhm1itmJBAA1CPeG5VfCKJYyqW4Egm5oQ== 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.5] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCTEuhmfwmeJhAAJHc2a25Tg83t25VlfBVVGw== 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.6] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCWEuhmqCYlLBAAi3vrfArZ09pvYaV1LHOThw== 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout:[osd.7] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQCZEuhmUgLyMBAA53W2ZBUCihxwDpjg/H3olQ== 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout:[client.0] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQB+Euhm1/GkNxAAlwXZfNtLEcrBIWX9lp3Nlg== 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2024-09-16T11:12:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQB+EuhmEGNQOhAAcDmdJ74pEPUs1BV0KTk8rw== 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr.y] 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQB+EuhmaAJBMhAAaCyw39s/Ehxo0JmIh2ICgQ== 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.0] 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCBEuhmI6IwIRAAW3jAroOUVxUiSrBRFTPHag== 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.1] 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCEEuhmTqkLKBAAl+3j8/Cy2roZM7JKMkjLsw== 2024-09-16T11:12:29.142 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.2] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCHEuhmxx2dKhAAc5ZeuxN2PkyjewfHdvDUrg== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.3] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCKEuhmH3YwMBAA13gnux6DEJiJiRR/D8+i1g== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr.x] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQB+EuhmzkIXNRAA5U/JfRsInu3sdQbPULjbxA== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.4] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCQEuhm1itmJBAA1CPeG5VfCKJYyqW4Egm5oQ== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.5] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCTEuhmfwmeJhAAJHc2a25Tg83t25VlfBVVGw== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.6] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCWEuhmqCYlLBAAi3vrfArZ09pvYaV1LHOThw== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.7] 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQCZEuhmUgLyMBAA53W2ZBUCihxwDpjg/H3olQ== 2024-09-16T11:12:29.143 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-09-16T11:12:29.144 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQB+Euhm1/GkNxAAlwXZfNtLEcrBIWX9lp3Nlg== 2024-09-16T11:12:29.144 INFO:teuthology.orchestra.run.smithi165.stdout:[client.1] 2024-09-16T11:12:29.144 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQB+EuhmEGNQOhAAcDmdJ74pEPUs1BV0KTk8rw== 2024-09-16T11:12:29.145 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:12:29.190 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:12:29.233 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.235 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.280 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.282 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.368 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.413 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.416 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.457 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:12:29.498 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:12:29.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.547 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.591 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.593 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.677 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.768 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:12:29.816 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:12:29.818 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:12:29.864 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:12:29.866 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:12:29.913 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T11:12:29.913 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T11:12:29.927 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:12:29.976 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T11:12:29.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T11:12:30.048 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:12:30.097 DEBUG:teuthology.orchestra.run.smithi162:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T11:12:30.109 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T11:12:30.126 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:12:30.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T11:12:30.236 DEBUG:teuthology.orchestra.run.smithi162:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:30.238 DEBUG:teuthology.orchestra.run.smithi165:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:12:30.243 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T11:12:30.244 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T11:12:30.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T11:12:30.286 INFO:tasks.ceph.mon.a:Started 2024-09-16T11:12:30.286 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T11:12:30.286 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T11:12:30.288 INFO:tasks.ceph.mon.c:Started 2024-09-16T11:12:30.289 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T11:12:30.289 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T11:12:30.291 INFO:tasks.ceph.mon.b:Started 2024-09-16T11:12:30.291 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T11:12:30.292 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T11:12:30.292 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T11:12:30.294 INFO:tasks.ceph.mgr.y:Started 2024-09-16T11:12:30.295 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T11:12:30.295 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T11:12:30.297 INFO:tasks.ceph.mgr.x:Started 2024-09-16T11:12:30.297 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T11:12:30.297 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T11:12:35.579 INFO:teuthology.orchestra.run.smithi162.stderr:adjusted tunables profile to default 2024-09-16T11:12:35.591 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T11:12:35.592 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:35.592 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T11:12:35.607 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:35.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T11:12:35.616 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:35.616 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T11:12:35.625 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-16T11:12:35.625 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T11:12:35.669 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:35.666+0000 7f22a5f67000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T11:12:35.683 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:35.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T11:12:35.700 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:35.700 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T11:12:35.760 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:35.760 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T11:12:35.769 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-16T11:12:35.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T11:12:35.779 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new b772f030-6c52-46c6-a992-36cadd7f9edd 0 2024-09-16T11:12:35.989 INFO:teuthology.orchestra.run.smithi165.stdout:0 2024-09-16T11:12:36.002 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 6335aa8f-a82b-45a4-be04-b1b064314f8d 1 2024-09-16T11:12:36.206 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-09-16T11:12:36.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 8d9432da-1713-44e1-9c96-db33f21cf333 2 2024-09-16T11:12:36.434 INFO:teuthology.orchestra.run.smithi165.stdout:2 2024-09-16T11:12:36.447 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new f17d5292-6b43-4887-a146-a647c60e4f86 3 2024-09-16T11:12:36.499 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:36.494+0000 7f22a5f67000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T11:12:36.556 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:36.550+0000 7f22a5f67000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T11:12:36.626 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:36.620+0000 7faff5394000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T11:12:36.653 INFO:teuthology.orchestra.run.smithi165.stdout:3 2024-09-16T11:12:36.667 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 93cb4050-ba1d-44c4-8e05-854a4cbb9236 4 2024-09-16T11:12:36.672 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:36.666+0000 7f22a5f67000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:12:36.869 INFO:teuthology.orchestra.run.smithi165.stdout:4 2024-09-16T11:12:36.883 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 432a0446-4f3b-436d-bba0-76184260e13c 5 2024-09-16T11:12:36.910 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:36.906+0000 7f22a5f67000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T11:12:36.962 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:36.958+0000 7f22a5f67000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T11:12:37.110 INFO:teuthology.orchestra.run.smithi165.stdout:5 2024-09-16T11:12:37.127 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 7c7e65e2-ab06-47f4-8d6d-712aa308e13f 6 2024-09-16T11:12:37.157 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:37.154+0000 7f22a5f67000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T11:12:37.222 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:37.218+0000 7f22a5f67000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T11:12:37.289 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:37.286+0000 7f22a5f67000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T11:12:37.329 INFO:teuthology.orchestra.run.smithi165.stdout:6 2024-09-16T11:12:37.346 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 827daf60-a01e-40a5-9595-7ec5406e4a8a 7 2024-09-16T11:12:37.415 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:37.408+0000 7faff5394000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T11:12:37.473 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:37.468+0000 7faff5394000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T11:12:37.542 INFO:teuthology.orchestra.run.smithi165.stdout:7 2024-09-16T11:12:37.557 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T11:12:37.557 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T11:12:37.560 INFO:tasks.ceph.osd.0:Started 2024-09-16T11:12:37.560 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T11:12:37.560 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T11:12:37.564 INFO:tasks.ceph.osd.1:Started 2024-09-16T11:12:37.564 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T11:12:37.564 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T11:12:37.566 INFO:tasks.ceph.osd.2:Started 2024-09-16T11:12:37.566 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T11:12:37.566 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T11:12:37.568 INFO:tasks.ceph.osd.3:Started 2024-09-16T11:12:37.568 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T11:12:37.568 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T11:12:37.570 INFO:tasks.ceph.osd.4:Started 2024-09-16T11:12:37.571 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T11:12:37.571 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T11:12:37.573 INFO:tasks.ceph.osd.5:Started 2024-09-16T11:12:37.573 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T11:12:37.573 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T11:12:37.577 INFO:tasks.ceph.osd.6:Started 2024-09-16T11:12:37.577 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T11:12:37.577 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T11:12:37.580 INFO:tasks.ceph.osd.7:Started 2024-09-16T11:12:37.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:12:37.605 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:37.600+0000 7faff5394000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:12:37.727 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:37.722+0000 7f22a5f67000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T11:12:37.768 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:12:37.769 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":10,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:12:37.534334+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:12:37.534334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:12:37.784 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T11:12:37.784 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T11:12:37.850 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:37.846+0000 7f22a5f67000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T11:12:37.857 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:37.852+0000 7faff5394000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T11:12:37.910 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:37.904+0000 7faff5394000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T11:12:37.934 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:12:37.930+0000 7fb5be250540 -1 Falling back to public interface 2024-09-16T11:12:37.950 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:12:37.946+0000 7f16797af540 -1 Falling back to public interface 2024-09-16T11:12:37.978 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:12:37.974+0000 7f237e0a2540 -1 Falling back to public interface 2024-09-16T11:12:38.084 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:38.076+0000 7faff5394000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T11:12:38.144 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:38.136+0000 7faff5394000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T11:12:38.171 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.166+0000 7f22a5f67000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T11:12:38.194 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:12:38.188+0000 7f08b1dfc540 -1 Falling back to public interface 2024-09-16T11:12:38.198 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:12:38.194+0000 7f0948814540 -1 Falling back to public interface 2024-09-16T11:12:38.203 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:12:38.196+0000 7fe520684540 -1 Falling back to public interface 2024-09-16T11:12:38.206 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:38.200+0000 7faff5394000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T11:12:38.219 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:12:38.212+0000 7fad960d6540 -1 Falling back to public interface 2024-09-16T11:12:38.219 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:12:38.212+0000 7fa4f805b540 -1 Falling back to public interface 2024-09-16T11:12:38.348 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.342+0000 7f22a5f67000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:12:38.421 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.418+0000 7f22a5f67000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T11:12:38.479 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.474+0000 7f22a5f67000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T11:12:38.540 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.534+0000 7f22a5f67000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T11:12:38.592 DEBUG:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:12:38.608 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:38.600+0000 7faff5394000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T11:12:38.720 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:38.712+0000 7faff5394000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T11:12:38.723 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.718+0000 7f22a5f67000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T11:12:38.760 INFO:teuthology.misc.health.smithi162.stdout: 2024-09-16T11:12:38.760 INFO:teuthology.misc.health.smithi162.stdout:{"epoch":10,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:12:37.534334+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:12:37.534334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:12:38.778 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T11:12:38.902 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.898+0000 7f22a5f67000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T11:12:38.968 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:38.962+0000 7f22a5f67000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T11:12:39.010 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.004+0000 7faff5394000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T11:12:39.139 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:39.134+0000 7f22a5f67000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T11:12:39.172 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.164+0000 7faff5394000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:12:39.208 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:39.202+0000 7f22a5f67000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T11:12:39.235 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.228+0000 7faff5394000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T11:12:39.288 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.280+0000 7faff5394000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T11:12:39.343 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.336+0000 7faff5394000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T11:12:39.512 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.504+0000 7faff5394000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T11:12:39.679 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.672+0000 7faff5394000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T11:12:39.744 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.736+0000 7faff5394000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T11:12:39.898 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.892+0000 7faff5394000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T11:12:39.911 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:39.906+0000 7f22a5f67000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T11:12:39.966 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:39.960+0000 7faff5394000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T11:12:40.671 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-09-16T11:12:40.664+0000 7faff5394000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T11:12:40.952 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:40.946+0000 7f228efd7700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T11:12:40.953 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:12:40.950+0000 7f228efd7700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T11:12:41.509 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:12:41.506+0000 7fb5be250540 -1 osd.1 0 log_to_monitors true 2024-09-16T11:12:41.769 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:12:41.766+0000 7f16797af540 -1 osd.3 0 log_to_monitors true 2024-09-16T11:12:41.773 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:12:41.770+0000 7f237e0a2540 -1 osd.2 0 log_to_monitors true 2024-09-16T11:12:41.889 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:12:41.884+0000 7fad960d6540 -1 osd.6 0 log_to_monitors true 2024-09-16T11:12:42.018 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:12:42.014+0000 7f0948814540 -1 osd.0 0 log_to_monitors true 2024-09-16T11:12:42.026 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:12:42.020+0000 7f08b1dfc540 -1 osd.4 0 log_to_monitors true 2024-09-16T11:12:42.108 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:12:42.100+0000 7fe520684540 -1 osd.5 0 log_to_monitors true 2024-09-16T11:12:42.131 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:12:42.124+0000 7fa4f805b540 -1 osd.7 0 log_to_monitors true 2024-09-16T11:12:45.584 DEBUG:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:12:45.644 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:12:45.636+0000 7fad91876700 -1 osd.6 0 waiting for initial osdmap 2024-09-16T11:12:45.649 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:12:45.644+0000 7fad8a619700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.666 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:12:45.660+0000 7f08aedc1700 -1 osd.4 0 waiting for initial osdmap 2024-09-16T11:12:45.671 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:12:45.664+0000 7f08a633f700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.744 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:12:45.738+0000 7f237b067700 -1 osd.2 0 waiting for initial osdmap 2024-09-16T11:12:45.752 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:12:45.750+0000 7f1674f4f700 -1 osd.3 0 waiting for initial osdmap 2024-09-16T11:12:45.754 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:12:45.750+0000 7f0943fb4700 -1 osd.0 0 waiting for initial osdmap 2024-09-16T11:12:45.756 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:12:45.750+0000 7f23725e5700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.760 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:12:45.754+0000 7f093cd57700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.762 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:12:45.758+0000 7f166dcf2700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.774 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:12:45.770+0000 7fb5b99f0700 -1 osd.1 0 waiting for initial osdmap 2024-09-16T11:12:45.780 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:12:45.774+0000 7fb5b2793700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.781 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:12:45.772+0000 7fe51be24700 -1 osd.5 0 waiting for initial osdmap 2024-09-16T11:12:45.785 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:12:45.780+0000 7fe514bc7700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.806 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:12:45.800+0000 7fa4f37fb700 -1 osd.7 0 waiting for initial osdmap 2024-09-16T11:12:45.812 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:12:45.804+0000 7fa4ec59e700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:12:45.912 INFO:teuthology.misc.health.smithi162.stdout: 2024-09-16T11:12:45.913 INFO:teuthology.misc.health.smithi162.stdout:{"epoch":12,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:12:43.499891+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:12:37.534334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:12:45.924 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T11:12:52.731 DEBUG:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:12:53.013 INFO:teuthology.misc.health.smithi162.stdout: 2024-09-16T11:12:53.013 INFO:teuthology.misc.health.smithi162.stdout:{"epoch":17,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:12:50.773046+0000","last_up_change":"2024-09-16T11:12:46.645944+0000","last_in_change":"2024-09-16T11:12:37.534334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":22378},{"type":"v1","addr":"172.21.15.165:6820","nonce":22378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":22378},{"type":"v1","addr":"172.21.15.165:6825","nonce":22378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":22378},{"type":"v1","addr":"172.21.15.165:6831","nonce":22378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":22378},{"type":"v1","addr":"172.21.15.165:6829","nonce":22378}]},"public_addr":"172.21.15.165:6820/22378","cluster_addr":"172.21.15.165:6825/22378","heartbeat_back_addr":"172.21.15.165:6831/22378","heartbeat_front_addr":"172.21.15.165:6829/22378","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:12:53.025 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T11:12:53.025 INFO:tasks.ceph:Creating RBD pool 2024-09-16T11:12:53.026 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T11:12:53.818 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'rbd' created 2024-09-16T11:12:53.829 DEBUG:teuthology.orchestra.run.smithi162:> rbd --cluster ceph pool init rbd 2024-09-16T11:12:56.850 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T11:12:56.851 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T11:12:56.851 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T11:12:56.851 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T11:12:56.852 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T11:12:57.188 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:12:57.203 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":22084},{"type":"v1","addr":"172.21.15.162:6833","nonce":22084}]},"active_addr":"172.21.15.162:6833/22084","active_change":"2024-09-16T11:12:39.929825+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4124,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":3595325100}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":622243646}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":4073969618}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.162:0","nonce":841530221}]}]} 2024-09-16T11:12:57.207 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T11:12:57.207 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T11:12:57.207 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:12:57.500 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:12:57.500 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:12:56.822648+0000","last_up_change":"2024-09-16T11:12:46.645944+0000","last_in_change":"2024-09-16T11:12:37.534334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:12:53.361644+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":22378},{"type":"v1","addr":"172.21.15.165:6820","nonce":22378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":22378},{"type":"v1","addr":"172.21.15.165:6825","nonce":22378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":22378},{"type":"v1","addr":"172.21.15.165:6831","nonce":22378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":22378},{"type":"v1","addr":"172.21.15.165:6829","nonce":22378}]},"public_addr":"172.21.15.165:6820/22378","cluster_addr":"172.21.15.165:6825/22378","heartbeat_back_addr":"172.21.15.165:6831/22378","heartbeat_front_addr":"172.21.15.165:6829/22378","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:12:57.514 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T11:12:57.515 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:12:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:12:57.800 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":21,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:12:56.822648+0000","last_up_change":"2024-09-16T11:12:46.645944+0000","last_in_change":"2024-09-16T11:12:37.534334+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:12:53.361644+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":22378},{"type":"v1","addr":"172.21.15.165:6820","nonce":22378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":22378},{"type":"v1","addr":"172.21.15.165:6825","nonce":22378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":22378},{"type":"v1","addr":"172.21.15.165:6831","nonce":22378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":22378},{"type":"v1","addr":"172.21.15.165:6829","nonce":22378}]},"public_addr":"172.21.15.165:6820/22378","cluster_addr":"172.21.15.165:6825/22378","heartbeat_back_addr":"172.21.15.165:6831/22378","heartbeat_front_addr":"172.21.15.165:6829/22378","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:12:57.820 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T11:12:57.820 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T11:12:57.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T11:12:57.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T11:12:57.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T11:12:57.822 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T11:12:57.823 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T11:12:57.823 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T11:12:58.021 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:12:58.021 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:12:58.024 INFO:teuthology.orchestra.run.smithi162.stdout:55834574855 2024-09-16T11:12:58.024 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:12:58.028 INFO:teuthology.orchestra.run.smithi162.stdout:55834574853 2024-09-16T11:12:58.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:12:58.032 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:12:58.032 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:12:58.033 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:12:58.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:12:58.044 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:12:58.044 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:12:58.045 INFO:teuthology.orchestra.run.smithi162.stdout:55834574853 2024-09-16T11:12:58.045 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:12:58.049 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:12:58.049 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:12:58.467 INFO:teuthology.orchestra.run.smithi162.stdout:55834574851 2024-09-16T11:12:58.479 INFO:teuthology.orchestra.run.smithi162.stdout:55834574853 2024-09-16T11:12:58.484 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.0 2024-09-16T11:12:58.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.7 2024-09-16T11:12:58.503 INFO:teuthology.orchestra.run.smithi162.stdout:55834574852 2024-09-16T11:12:58.511 INFO:teuthology.orchestra.run.smithi162.stdout:55834574852 2024-09-16T11:12:58.512 INFO:teuthology.orchestra.run.smithi162.stdout:55834574852 2024-09-16T11:12:58.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2024-09-16T11:12:58.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.4 2024-09-16T11:12:58.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2024-09-16T11:12:58.532 INFO:teuthology.orchestra.run.smithi162.stdout:55834574851 2024-09-16T11:12:58.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.6 2024-09-16T11:12:58.561 INFO:teuthology.orchestra.run.smithi162.stdout:55834574852 2024-09-16T11:12:58.570 INFO:teuthology.orchestra.run.smithi162.stdout:55834574852 2024-09-16T11:12:58.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.2 2024-09-16T11:12:58.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2024-09-16T11:12:59.485 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:12:59.498 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:12:59.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:12:59.526 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:12:59.528 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:12:59.544 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:12:59.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:12:59.592 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:12:59.925 INFO:teuthology.orchestra.run.smithi162.stdout:55834574852 2024-09-16T11:12:59.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-09-16T11:12:59.953 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:12:59.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-09-16T11:12:59.971 INFO:teuthology.orchestra.run.smithi162.stdout:55834574853 2024-09-16T11:12:59.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-09-16T11:13:00.011 INFO:teuthology.orchestra.run.smithi162.stdout:55834574853 2024-09-16T11:13:00.021 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:13:00.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-09-16T11:13:00.029 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:00.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-09-16T11:13:00.033 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:00.034 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:13:00.045 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:13:00.049 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:13:00.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-09-16T11:13:00.050 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:00.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-09-16T11:13:00.058 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:00.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-09-16T11:13:00.064 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:00.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:13:00.971 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:13:00.986 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:13:01.250 INFO:teuthology.orchestra.run.smithi162.stdout:55834574853 2024-09-16T11:13:01.257 INFO:teuthology.orchestra.run.smithi162.stdout:55834574855 2024-09-16T11:13:01.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-09-16T11:13:01.268 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:01.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-09-16T11:13:01.272 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:01.294 INFO:teuthology.orchestra.run.smithi162.stdout:55834574854 2024-09-16T11:13:01.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-09-16T11:13:01.307 DEBUG:teuthology.parallel:result is None 2024-09-16T11:13:01.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:13:01.307 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:13:01.576 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:01.579 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:13:01.589 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-16T11:13:00.399132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381708,"kb_used_data":2852,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593012,"statfs":{"total":773094113280,"available":770655244288,"internally_reserved":0,"allocated":2920448,"data_stored":1761154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.584318"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.908350+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:12:56.908350+0000","last_peered":"2024-09-16T11:12:56.908350+0000","last_clean":"2024-09-16T11:12:56.908350+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:12:56.908350+0000","last_undegraded":"2024-09-16T11:12:56.908350+0000","last_fullsized":"2024-09-16T11:12:56.908350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.845531+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:12:56.845531+0000","last_peered":"2024-09-16T11:12:56.845531+0000","last_clean":"2024-09-16T11:12:56.845531+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:12:56.845531+0000","last_undegraded":"2024-09-16T11:12:56.845531+0000","last_fullsized":"2024-09-16T11:12:56.845531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:57.281556+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:12:57.281556+0000","last_peered":"2024-09-16T11:12:57.281556+0000","last_clean":"2024-09-16T11:12:57.281556+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:12:57.281556+0000","last_undegraded":"2024-09-16T11:12:57.281556+0000","last_fullsized":"2024-09-16T11:12:57.281556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.845434+0000","last_change":"2024-09-16T11:12:56.845864+0000","last_active":"2024-09-16T11:12:56.845434+0000","last_peered":"2024-09-16T11:12:56.845434+0000","last_clean":"2024-09-16T11:12:56.845434+0000","last_became_active":"2024-09-16T11:12:55.087737+0000","last_became_peered":"2024-09-16T11:12:55.087737+0000","last_unstale":"2024-09-16T11:12:56.845434+0000","last_undegraded":"2024-09-16T11:12:56.845434+0000","last_fullsized":"2024-09-16T11:12:56.845434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:12.366691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057656500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.834191+0000","last_change":"2024-09-16T11:12:56.834191+0000","last_active":"2024-09-16T11:12:56.834191+0000","last_peered":"2024-09-16T11:12:56.834191+0000","last_clean":"2024-09-16T11:12:56.834191+0000","last_became_active":"2024-09-16T11:12:54.819534+0000","last_became_peered":"2024-09-16T11:12:54.819534+0000","last_unstale":"2024-09-16T11:12:56.834191+0000","last_undegraded":"2024-09-16T11:12:56.834191+0000","last_fullsized":"2024-09-16T11:12:56.834191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:47.418906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.838005+0000","last_change":"2024-09-16T11:12:56.838315+0000","last_active":"2024-09-16T11:12:56.838005+0000","last_peered":"2024-09-16T11:12:56.838005+0000","last_clean":"2024-09-16T11:12:56.838005+0000","last_became_active":"2024-09-16T11:12:54.831617+0000","last_became_peered":"2024-09-16T11:12:54.831617+0000","last_unstale":"2024-09-16T11:12:56.838005+0000","last_undegraded":"2024-09-16T11:12:56.838005+0000","last_fullsized":"2024-09-16T11:12:56.838005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:16.617321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044984799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:57.281391+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:12:57.281391+0000","last_peered":"2024-09-16T11:12:57.281391+0000","last_clean":"2024-09-16T11:12:57.281391+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:12:57.281391+0000","last_undegraded":"2024-09-16T11:12:57.281391+0000","last_fullsized":"2024-09-16T11:12:57.281391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.833444+0000","last_change":"2024-09-16T11:12:56.833841+0000","last_active":"2024-09-16T11:12:56.833444+0000","last_peered":"2024-09-16T11:12:56.833444+0000","last_clean":"2024-09-16T11:12:56.833444+0000","last_became_active":"2024-09-16T11:12:54.826068+0000","last_became_peered":"2024-09-16T11:12:54.826068+0000","last_unstale":"2024-09-16T11:12:56.833444+0000","last_undegraded":"2024-09-16T11:12:56.833444+0000","last_fullsized":"2024-09-16T11:12:56.833444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.525257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":103,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T11:12:55.932871+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:12:55.932871+0000","last_peered":"2024-09-16T11:12:55.932871+0000","last_clean":"2024-09-16T11:12:55.932871+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:12:55.932871+0000","last_undegraded":"2024-09-16T11:12:55.932871+0000","last_fullsized":"2024-09-16T11:12:55.932871+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298136,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073704,"statfs":{"total":96636764160,"available":96331472896,"internally_reserved":0,"allocated":798720,"data_stored":657823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":73294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":73294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":811008,"data_stored":663643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:13:01.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:13:01.855 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:01.858 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:13:01.868 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-16T11:13:00.399132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2381708,"kb_used_data":2852,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593012,"statfs":{"total":773094113280,"available":770655244288,"internally_reserved":0,"allocated":2920448,"data_stored":1761154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.584318"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.908350+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:12:56.908350+0000","last_peered":"2024-09-16T11:12:56.908350+0000","last_clean":"2024-09-16T11:12:56.908350+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:12:56.908350+0000","last_undegraded":"2024-09-16T11:12:56.908350+0000","last_fullsized":"2024-09-16T11:12:56.908350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.845531+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:12:56.845531+0000","last_peered":"2024-09-16T11:12:56.845531+0000","last_clean":"2024-09-16T11:12:56.845531+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:12:56.845531+0000","last_undegraded":"2024-09-16T11:12:56.845531+0000","last_fullsized":"2024-09-16T11:12:56.845531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:57.281556+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:12:57.281556+0000","last_peered":"2024-09-16T11:12:57.281556+0000","last_clean":"2024-09-16T11:12:57.281556+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:12:57.281556+0000","last_undegraded":"2024-09-16T11:12:57.281556+0000","last_fullsized":"2024-09-16T11:12:57.281556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.845434+0000","last_change":"2024-09-16T11:12:56.845864+0000","last_active":"2024-09-16T11:12:56.845434+0000","last_peered":"2024-09-16T11:12:56.845434+0000","last_clean":"2024-09-16T11:12:56.845434+0000","last_became_active":"2024-09-16T11:12:55.087737+0000","last_became_peered":"2024-09-16T11:12:55.087737+0000","last_unstale":"2024-09-16T11:12:56.845434+0000","last_undegraded":"2024-09-16T11:12:56.845434+0000","last_fullsized":"2024-09-16T11:12:56.845434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:12.366691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057656500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.834191+0000","last_change":"2024-09-16T11:12:56.834191+0000","last_active":"2024-09-16T11:12:56.834191+0000","last_peered":"2024-09-16T11:12:56.834191+0000","last_clean":"2024-09-16T11:12:56.834191+0000","last_became_active":"2024-09-16T11:12:54.819534+0000","last_became_peered":"2024-09-16T11:12:54.819534+0000","last_unstale":"2024-09-16T11:12:56.834191+0000","last_undegraded":"2024-09-16T11:12:56.834191+0000","last_fullsized":"2024-09-16T11:12:56.834191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:47.418906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.838005+0000","last_change":"2024-09-16T11:12:56.838315+0000","last_active":"2024-09-16T11:12:56.838005+0000","last_peered":"2024-09-16T11:12:56.838005+0000","last_clean":"2024-09-16T11:12:56.838005+0000","last_became_active":"2024-09-16T11:12:54.831617+0000","last_became_peered":"2024-09-16T11:12:54.831617+0000","last_unstale":"2024-09-16T11:12:56.838005+0000","last_undegraded":"2024-09-16T11:12:56.838005+0000","last_fullsized":"2024-09-16T11:12:56.838005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:16.617321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044984799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:57.281391+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:12:57.281391+0000","last_peered":"2024-09-16T11:12:57.281391+0000","last_clean":"2024-09-16T11:12:57.281391+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:12:57.281391+0000","last_undegraded":"2024-09-16T11:12:57.281391+0000","last_fullsized":"2024-09-16T11:12:57.281391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T11:12:56.833444+0000","last_change":"2024-09-16T11:12:56.833841+0000","last_active":"2024-09-16T11:12:56.833444+0000","last_peered":"2024-09-16T11:12:56.833444+0000","last_clean":"2024-09-16T11:12:56.833444+0000","last_became_active":"2024-09-16T11:12:54.826068+0000","last_became_peered":"2024-09-16T11:12:54.826068+0000","last_unstale":"2024-09-16T11:12:56.833444+0000","last_undegraded":"2024-09-16T11:12:56.833444+0000","last_fullsized":"2024-09-16T11:12:56.833444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.525257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":103,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T11:12:55.932871+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:12:55.932871+0000","last_peered":"2024-09-16T11:12:55.932871+0000","last_clean":"2024-09-16T11:12:55.932871+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:12:55.932871+0000","last_undegraded":"2024-09-16T11:12:55.932871+0000","last_fullsized":"2024-09-16T11:12:55.932871+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298136,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073704,"statfs":{"total":96636764160,"available":96331472896,"internally_reserved":0,"allocated":798720,"data_stored":657823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":73294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.111}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":73275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297572,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074268,"statfs":{"total":96636764160,"available":96332050432,"internally_reserved":0,"allocated":221184,"data_stored":73294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":811008,"data_stored":663643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:13:01.869 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:13:01.869 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T11:13:01.869 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T11:13:01.869 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T11:13:02.170 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:02.170 INFO:teuthology.orchestra.run.smithi162.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T11:13:02.183 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T11:13:02.183 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T11:13:02.193 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T11:13:02.193 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:13:02.193 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2024-09-16T11:13:03.025 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cache' created 2024-09-16T11:13:03.037 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2024-09-16T11:13:04.026 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2024-09-16T11:13:04.039 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2024-09-16T11:13:05.040 INFO:teuthology.orchestra.run.smithi162.stderr:set cache-mode for pool 'cache' to writeback 2024-09-16T11:13:05.051 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2024-09-16T11:13:06.049 INFO:teuthology.orchestra.run.smithi162.stderr:overlay for 'rbd' is now (or already was) 'cache' 2024-09-16T11:13:06.063 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2024-09-16T11:13:07.064 INFO:teuthology.orchestra.run.smithi162.stderr:set pool 3 hit_set_type to bloom 2024-09-16T11:13:07.078 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2024-09-16T11:13:08.068 INFO:teuthology.orchestra.run.smithi162.stderr:set pool 3 hit_set_count to 8 2024-09-16T11:13:08.082 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2024-09-16T11:13:09.073 INFO:teuthology.orchestra.run.smithi162.stderr:set pool 3 hit_set_period to 60 2024-09-16T11:13:09.085 DEBUG:teuthology.orchestra.run.smithi162:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2024-09-16T11:13:10.086 INFO:teuthology.orchestra.run.smithi162.stderr:set pool 3 target_max_objects to 250 2024-09-16T11:13:10.101 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T11:13:10.111 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T11:13:10.112 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'mons down', 'mon down', 'out of quorum'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T11:13:10.112 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T11:13:10.112 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T11:13:10.430 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T11:13:10.430 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:13:10.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:13:10.715 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:10.718 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:13:10.729 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-16T11:13:10.076855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382164,"kb_used_data":3296,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592556,"statfs":{"total":773094113280,"available":770654777344,"internally_reserved":0,"allocated":3375104,"data_stored":1985548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.036597"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116262+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:13:04.116262+0000","last_peered":"2024-09-16T11:13:04.116262+0000","last_clean":"2024-09-16T11:13:04.116262+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:13:04.116262+0000","last_undegraded":"2024-09-16T11:13:04.116262+0000","last_fullsized":"2024-09-16T11:13:04.116262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.039265+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:13:05.039265+0000","last_peered":"2024-09-16T11:13:05.039265+0000","last_clean":"2024-09-16T11:13:05.039265+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:13:05.039265+0000","last_undegraded":"2024-09-16T11:13:05.039265+0000","last_fullsized":"2024-09-16T11:13:05.039265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116067+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:13:04.116067+0000","last_peered":"2024-09-16T11:13:04.116067+0000","last_clean":"2024-09-16T11:13:04.116067+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:13:04.116067+0000","last_undegraded":"2024-09-16T11:13:04.116067+0000","last_fullsized":"2024-09-16T11:13:04.116067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.039270+0000","last_change":"2024-09-16T11:12:56.845864+0000","last_active":"2024-09-16T11:13:05.039270+0000","last_peered":"2024-09-16T11:13:05.039270+0000","last_clean":"2024-09-16T11:13:05.039270+0000","last_became_active":"2024-09-16T11:12:55.087737+0000","last_became_peered":"2024-09-16T11:12:55.087737+0000","last_unstale":"2024-09-16T11:13:05.039270+0000","last_undegraded":"2024-09-16T11:13:05.039270+0000","last_fullsized":"2024-09-16T11:13:05.039270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:12.366691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057656500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T11:13:06.040633+0000","last_change":"2024-09-16T11:13:04.021351+0000","last_active":"2024-09-16T11:13:06.040633+0000","last_peered":"2024-09-16T11:13:06.040633+0000","last_clean":"2024-09-16T11:13:06.040633+0000","last_became_active":"2024-09-16T11:13:04.020623+0000","last_became_peered":"2024-09-16T11:13:04.020623+0000","last_unstale":"2024-09-16T11:13:06.040633+0000","last_undegraded":"2024-09-16T11:13:06.040633+0000","last_fullsized":"2024-09-16T11:13:06.040633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:59.977907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T11:13:06.038749+0000","last_change":"2024-09-16T11:12:56.834191+0000","last_active":"2024-09-16T11:13:06.038749+0000","last_peered":"2024-09-16T11:13:06.038749+0000","last_clean":"2024-09-16T11:13:06.038749+0000","last_became_active":"2024-09-16T11:12:54.819534+0000","last_became_peered":"2024-09-16T11:12:54.819534+0000","last_unstale":"2024-09-16T11:13:06.038749+0000","last_undegraded":"2024-09-16T11:13:06.038749+0000","last_fullsized":"2024-09-16T11:13:06.038749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:47.418906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.039302+0000","last_change":"2024-09-16T11:13:04.123661+0000","last_active":"2024-09-16T11:13:05.039302+0000","last_peered":"2024-09-16T11:13:05.039302+0000","last_clean":"2024-09-16T11:13:05.039302+0000","last_became_active":"2024-09-16T11:13:04.123099+0000","last_became_peered":"2024-09-16T11:13:04.123099+0000","last_unstale":"2024-09-16T11:13:05.039302+0000","last_undegraded":"2024-09-16T11:13:05.039302+0000","last_fullsized":"2024-09-16T11:13:05.039302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:46:32.328475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.119367+0000","last_change":"2024-09-16T11:12:56.838315+0000","last_active":"2024-09-16T11:13:04.119367+0000","last_peered":"2024-09-16T11:13:04.119367+0000","last_clean":"2024-09-16T11:13:04.119367+0000","last_became_active":"2024-09-16T11:12:54.831617+0000","last_became_peered":"2024-09-16T11:12:54.831617+0000","last_unstale":"2024-09-16T11:13:04.119367+0000","last_undegraded":"2024-09-16T11:13:04.119367+0000","last_fullsized":"2024-09-16T11:13:04.119367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:16.617321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044984799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.031010+0000","last_change":"2024-09-16T11:13:04.026433+0000","last_active":"2024-09-16T11:13:04.031010+0000","last_peered":"2024-09-16T11:13:04.031010+0000","last_clean":"2024-09-16T11:13:04.031010+0000","last_became_active":"2024-09-16T11:13:04.026067+0000","last_became_peered":"2024-09-16T11:13:04.026067+0000","last_unstale":"2024-09-16T11:13:04.031010+0000","last_undegraded":"2024-09-16T11:13:04.031010+0000","last_fullsized":"2024-09-16T11:13:04.031010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:17.947533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116043+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:13:04.116043+0000","last_peered":"2024-09-16T11:13:04.116043+0000","last_clean":"2024-09-16T11:13:04.116043+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:13:04.116043+0000","last_undegraded":"2024-09-16T11:13:04.116043+0000","last_fullsized":"2024-09-16T11:13:04.116043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.038845+0000","last_change":"2024-09-16T11:13:04.025567+0000","last_active":"2024-09-16T11:13:05.038845+0000","last_peered":"2024-09-16T11:13:05.038845+0000","last_clean":"2024-09-16T11:13:05.038845+0000","last_became_active":"2024-09-16T11:13:04.025252+0000","last_became_peered":"2024-09-16T11:13:04.025252+0000","last_unstale":"2024-09-16T11:13:05.038845+0000","last_undegraded":"2024-09-16T11:13:05.038845+0000","last_fullsized":"2024-09-16T11:13:05.038845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:12.470032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T11:13:06.038726+0000","last_change":"2024-09-16T11:12:56.833841+0000","last_active":"2024-09-16T11:13:06.038726+0000","last_peered":"2024-09-16T11:13:06.038726+0000","last_clean":"2024-09-16T11:13:06.038726+0000","last_became_active":"2024-09-16T11:12:54.826068+0000","last_became_peered":"2024-09-16T11:12:54.826068+0000","last_unstale":"2024-09-16T11:13:06.038726+0000","last_undegraded":"2024-09-16T11:13:06.038726+0000","last_fullsized":"2024-09-16T11:13:06.038726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.525257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116050+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:13:04.116050+0000","last_peered":"2024-09-16T11:13:04.116050+0000","last_clean":"2024-09-16T11:13:04.116050+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:13:04.116050+0000","last_undegraded":"2024-09-16T11:13:04.116050+0000","last_fullsized":"2024-09-16T11:13:04.116050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":860160,"data_stored":687779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":266240,"data_stored":97411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":294912,"data_stored":110026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":278528,"data_stored":103856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":266240,"data_stored":97411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":266240,"data_stored":97411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":282624,"data_stored":103875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":860160,"data_stored":687779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:13:10.731 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:13:11.030 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:11.033 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:13:11.044 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-16T11:13:10.076855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382164,"kb_used_data":3296,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752592556,"statfs":{"total":773094113280,"available":770654777344,"internally_reserved":0,"allocated":3375104,"data_stored":1985548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.036597"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116262+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:13:04.116262+0000","last_peered":"2024-09-16T11:13:04.116262+0000","last_clean":"2024-09-16T11:13:04.116262+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:13:04.116262+0000","last_undegraded":"2024-09-16T11:13:04.116262+0000","last_fullsized":"2024-09-16T11:13:04.116262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.039265+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:13:05.039265+0000","last_peered":"2024-09-16T11:13:05.039265+0000","last_clean":"2024-09-16T11:13:05.039265+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:13:05.039265+0000","last_undegraded":"2024-09-16T11:13:05.039265+0000","last_fullsized":"2024-09-16T11:13:05.039265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116067+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:13:04.116067+0000","last_peered":"2024-09-16T11:13:04.116067+0000","last_clean":"2024-09-16T11:13:04.116067+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:13:04.116067+0000","last_undegraded":"2024-09-16T11:13:04.116067+0000","last_fullsized":"2024-09-16T11:13:04.116067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.039270+0000","last_change":"2024-09-16T11:12:56.845864+0000","last_active":"2024-09-16T11:13:05.039270+0000","last_peered":"2024-09-16T11:13:05.039270+0000","last_clean":"2024-09-16T11:13:05.039270+0000","last_became_active":"2024-09-16T11:12:55.087737+0000","last_became_peered":"2024-09-16T11:12:55.087737+0000","last_unstale":"2024-09-16T11:13:05.039270+0000","last_undegraded":"2024-09-16T11:13:05.039270+0000","last_fullsized":"2024-09-16T11:13:05.039270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:12.366691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057656500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T11:13:06.040633+0000","last_change":"2024-09-16T11:13:04.021351+0000","last_active":"2024-09-16T11:13:06.040633+0000","last_peered":"2024-09-16T11:13:06.040633+0000","last_clean":"2024-09-16T11:13:06.040633+0000","last_became_active":"2024-09-16T11:13:04.020623+0000","last_became_peered":"2024-09-16T11:13:04.020623+0000","last_unstale":"2024-09-16T11:13:06.040633+0000","last_undegraded":"2024-09-16T11:13:06.040633+0000","last_fullsized":"2024-09-16T11:13:06.040633+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:59.977907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T11:13:06.038749+0000","last_change":"2024-09-16T11:12:56.834191+0000","last_active":"2024-09-16T11:13:06.038749+0000","last_peered":"2024-09-16T11:13:06.038749+0000","last_clean":"2024-09-16T11:13:06.038749+0000","last_became_active":"2024-09-16T11:12:54.819534+0000","last_became_peered":"2024-09-16T11:12:54.819534+0000","last_unstale":"2024-09-16T11:13:06.038749+0000","last_undegraded":"2024-09-16T11:13:06.038749+0000","last_fullsized":"2024-09-16T11:13:06.038749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:47.418906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.039302+0000","last_change":"2024-09-16T11:13:04.123661+0000","last_active":"2024-09-16T11:13:05.039302+0000","last_peered":"2024-09-16T11:13:05.039302+0000","last_clean":"2024-09-16T11:13:05.039302+0000","last_became_active":"2024-09-16T11:13:04.123099+0000","last_became_peered":"2024-09-16T11:13:04.123099+0000","last_unstale":"2024-09-16T11:13:05.039302+0000","last_undegraded":"2024-09-16T11:13:05.039302+0000","last_fullsized":"2024-09-16T11:13:05.039302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:46:32.328475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.119367+0000","last_change":"2024-09-16T11:12:56.838315+0000","last_active":"2024-09-16T11:13:04.119367+0000","last_peered":"2024-09-16T11:13:04.119367+0000","last_clean":"2024-09-16T11:13:04.119367+0000","last_became_active":"2024-09-16T11:12:54.831617+0000","last_became_peered":"2024-09-16T11:12:54.831617+0000","last_unstale":"2024-09-16T11:13:04.119367+0000","last_undegraded":"2024-09-16T11:13:04.119367+0000","last_fullsized":"2024-09-16T11:13:04.119367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:16.617321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044984799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.031010+0000","last_change":"2024-09-16T11:13:04.026433+0000","last_active":"2024-09-16T11:13:04.031010+0000","last_peered":"2024-09-16T11:13:04.031010+0000","last_clean":"2024-09-16T11:13:04.031010+0000","last_became_active":"2024-09-16T11:13:04.026067+0000","last_became_peered":"2024-09-16T11:13:04.026067+0000","last_unstale":"2024-09-16T11:13:04.031010+0000","last_undegraded":"2024-09-16T11:13:04.031010+0000","last_fullsized":"2024-09-16T11:13:04.031010+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:17.947533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116043+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:13:04.116043+0000","last_peered":"2024-09-16T11:13:04.116043+0000","last_clean":"2024-09-16T11:13:04.116043+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:13:04.116043+0000","last_undegraded":"2024-09-16T11:13:04.116043+0000","last_fullsized":"2024-09-16T11:13:04.116043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-16T11:13:05.038845+0000","last_change":"2024-09-16T11:13:04.025567+0000","last_active":"2024-09-16T11:13:05.038845+0000","last_peered":"2024-09-16T11:13:05.038845+0000","last_clean":"2024-09-16T11:13:05.038845+0000","last_became_active":"2024-09-16T11:13:04.025252+0000","last_became_peered":"2024-09-16T11:13:04.025252+0000","last_unstale":"2024-09-16T11:13:05.038845+0000","last_undegraded":"2024-09-16T11:13:05.038845+0000","last_fullsized":"2024-09-16T11:13:05.038845+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:12.470032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-16T11:13:06.038726+0000","last_change":"2024-09-16T11:12:56.833841+0000","last_active":"2024-09-16T11:13:06.038726+0000","last_peered":"2024-09-16T11:13:06.038726+0000","last_clean":"2024-09-16T11:13:06.038726+0000","last_became_active":"2024-09-16T11:12:54.826068+0000","last_became_peered":"2024-09-16T11:12:54.826068+0000","last_unstale":"2024-09-16T11:13:06.038726+0000","last_undegraded":"2024-09-16T11:13:06.038726+0000","last_fullsized":"2024-09-16T11:13:06.038726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:40:24.525257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":107,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-16T11:13:04.116050+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:13:04.116050+0000","last_peered":"2024-09-16T11:13:04.116050+0000","last_clean":"2024-09-16T11:13:04.116050+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:13:04.116050+0000","last_undegraded":"2024-09-16T11:13:04.116050+0000","last_fullsized":"2024-09-16T11:13:04.116050+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":860160,"data_stored":687779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":266240,"data_stored":97411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":294912,"data_stored":110026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1659999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":278528,"data_stored":103856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":266240,"data_stored":97411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":266240,"data_stored":97411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":282624,"data_stored":103875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":860160,"data_stored":687779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.137}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:13:11.045 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:13:11.045 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T11:13:11.330 INFO:teuthology.orchestra.run.smithi162.stdout:epoch 30 2024-09-16T11:13:11.330 INFO:teuthology.orchestra.run.smithi162.stdout:fsid 7e3ff0c6-8090-4b11-95d6-fd611813de49 2024-09-16T11:13:11.330 INFO:teuthology.orchestra.run.smithi162.stdout:created 2024-09-16T11:12:35.459871+0000 2024-09-16T11:13:11.330 INFO:teuthology.orchestra.run.smithi162.stdout:modified 2024-09-16T11:13:10.068826+0000 2024-09-16T11:13:11.330 INFO:teuthology.orchestra.run.smithi162.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:crush_version 4 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:full_ratio 0.95 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:backfillfull_ratio 0.9 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:nearfull_ratio 0.85 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:require_min_compat_client luminous 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:min_compat_client jewel 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:require_osd_release quincy 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:stretch_mode_enabled false 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 26 lfor 26/26/26 flags hashpspool,selfmanaged_snaps tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:pool 3 'cache' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 4 pgp_num 4 autoscale_mode off last_change 30 lfor 26/26/26 flags hashpspool,incomplete_clones,selfmanaged_snaps tier_of 2 cache_mode writeback target_objects 250 hit_set bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 60s x8 decay_rate 0 search_last_n 0 stripe_width 0 application rbd 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:max_osd 8 2024-09-16T11:13:11.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6824/22248,v1:172.21.15.162:6825/22248] [v2:172.21.15.162:6826/22248,v1:172.21.15.162:6827/22248] exists,up b772f030-6c52-46c6-a992-36cadd7f9edd 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/22247,v1:172.21.15.162:6801/22247] [v2:172.21.15.162:6802/22247,v1:172.21.15.162:6803/22247] exists,up 6335aa8f-a82b-45a4-be04-b1b064314f8d 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/22258,v1:172.21.15.162:6817/22258] [v2:172.21.15.162:6818/22258,v1:172.21.15.162:6819/22258] exists,up 8d9432da-1713-44e1-9c96-db33f21cf333 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/22257,v1:172.21.15.162:6809/22257] [v2:172.21.15.162:6810/22257,v1:172.21.15.162:6811/22257] exists,up f17d5292-6b43-4887-a146-a647c60e4f86 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/22379,v1:172.21.15.165:6801/22379] [v2:172.21.15.165:6802/22379,v1:172.21.15.165:6803/22379] exists,up 93cb4050-ba1d-44c4-8e05-854a4cbb9236 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/22380,v1:172.21.15.165:6809/22380] [v2:172.21.15.165:6810/22380,v1:172.21.15.165:6811/22380] exists,up 432a0446-4f3b-436d-bba0-76184260e13c 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/22381,v1:172.21.15.165:6818/22381] [v2:172.21.15.165:6819/22381,v1:172.21.15.165:6821/22381] exists,up 7c7e65e2-ab06-47f4-8d6d-712aa308e13f 2024-09-16T11:13:11.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6817/22378,v1:172.21.15.165:6820/22378] [v2:172.21.15.165:6823/22378,v1:172.21.15.165:6825/22378] exists,up 827daf60-a01e-40a5-9595-7ec5406e4a8a 2024-09-16T11:13:11.343 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6824/22248,v1:172.21.15.162:6825/22248] [v2:172.21.15.162:6826/22248,v1:172.21.15.162:6827/22248] exists,up b772f030-6c52-46c6-a992-36cadd7f9edd', 'osd.1 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/22247,v1:172.21.15.162:6801/22247] [v2:172.21.15.162:6802/22247,v1:172.21.15.162:6803/22247] exists,up 6335aa8f-a82b-45a4-be04-b1b064314f8d', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/22258,v1:172.21.15.162:6817/22258] [v2:172.21.15.162:6818/22258,v1:172.21.15.162:6819/22258] exists,up 8d9432da-1713-44e1-9c96-db33f21cf333', 'osd.3 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/22257,v1:172.21.15.162:6809/22257] [v2:172.21.15.162:6810/22257,v1:172.21.15.162:6811/22257] exists,up f17d5292-6b43-4887-a146-a647c60e4f86', 'osd.4 up in weight 1 up_from 13 up_thru 23 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/22379,v1:172.21.15.165:6801/22379] [v2:172.21.15.165:6802/22379,v1:172.21.15.165:6803/22379] exists,up 93cb4050-ba1d-44c4-8e05-854a4cbb9236', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/22380,v1:172.21.15.165:6809/22380] [v2:172.21.15.165:6810/22380,v1:172.21.15.165:6811/22380] exists,up 432a0446-4f3b-436d-bba0-76184260e13c', 'osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/22381,v1:172.21.15.165:6818/22381] [v2:172.21.15.165:6819/22381,v1:172.21.15.165:6821/22381] exists,up 7c7e65e2-ab06-47f4-8d6d-712aa308e13f', 'osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6817/22378,v1:172.21.15.165:6820/22378] [v2:172.21.15.165:6823/22378,v1:172.21.15.165:6825/22378] exists,up 827daf60-a01e-40a5-9595-7ec5406e4a8a'] 2024-09-16T11:13:11.344 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T11:13:11.444 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:11.444 INFO:teuthology.orchestra.run.smithi162.stdout: "name": "mon.a", 2024-09-16T11:13:11.444 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster": "ceph", 2024-09-16T11:13:11.444 INFO:teuthology.orchestra.run.smithi162.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "admin_socket_mode": "", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_client_required": "cephx, none", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_cluster_required": "cephx", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_debug": "true", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_service_required": "cephx", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "auth_supported": "cephx", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_aio": "true", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T11:13:11.445 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_aio_reap_max": "16", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_async_discard": "false", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_block_size": "4096", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_debug_aio": "false", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_enable_discard": "false", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_flock_retry": "3", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_inject_crash": "0", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_ioring": "false", 2024-09-16T11:13:11.446 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_ioring_hipri": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bdev_type": "", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_allocator": "hybrid", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_buffered_io": "true", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T11:13:11.447 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_replay_recovery": "false", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_sync_write": "false", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T11:13:11.448 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_allocator": "hybrid", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_create": "true", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_db_create": "false", 2024-09-16T11:13:11.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_db_path": "", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_db_size": "0", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_path": "", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_size": "107374182400", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_wal_create": "false", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_wal_path": "", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_bluefs": "true", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T11:13:11.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_autotune": "true", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_size": "0", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T11:13:11.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_type": "2q", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_clone_cow": "true", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_mode": "none", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T11:13:11.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_freelist": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T11:13:11.453 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fail_eio": "false", 2024-09-16T11:13:11.454 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T11:13:11.455 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_max_blob_size": "0", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T11:13:11.456 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-09-16T11:13:11.457 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_spdk_mem": "512", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T11:13:11.458 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_tracing": "false", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T11:13:11.459 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_cluster_require_version": "2", 2024-09-16T11:13:11.460 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_require_signatures": "false", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_require_version": "2", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_service_require_signatures": "false", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_service_require_version": "2", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "cephx_sign_messages": "true", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "chdir": "", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_acl_type": "", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_asio_thread_count": "2", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_cache_mid": "0.750000", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_cache_size": "16384", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_caps_release_delay": "5", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_check_pool_perm": "true", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T11:13:11.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_debug_force_sync_read": "false", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_debug_getattr_caps": "false", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_debug_inject_features": "", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_die_on_failed_remount": "false", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_dirsize_rbytes": "true", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_force_lazyio": "false", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_fs": "", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_inject_release_failure": "false", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_max_inline_size": "4096", 2024-09-16T11:13:11.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_mds_namespace": "", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_metadata": "", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_mount_gid": "-1", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_mount_timeout": "300", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_mount_uid": "-1", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_mountpoint": "/", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_notify_timeout": "10", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_oc": "true", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_oc_max_objects": "1000", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_oc_size": "209715200", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_permissions": "true", 2024-09-16T11:13:11.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_quota": "true", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_quota_df": "true", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_readahead_max_bytes": "0", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_readahead_max_periods": "4", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_readahead_min": "131072", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_reconnect_stale": "false", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_shutdown_timeout": "30", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_snapdir": ".snap", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_tick_interval": "1", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_trace": "", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_use_faked_inos": "false", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_use_random_mds": "false", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_graylog": "false", 2024-09-16T11:13:11.464 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_graylog_port": "12201", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_monitors": "default=true", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_syslog": "false", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "clog_to_syslog_level": "info", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "-", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_network": "", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_network_interface": "", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "compressor_zlib_isal": "false", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "compressor_zlib_level": "5", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "compressor_zstd_level": "1", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T11:13:11.465 INFO:teuthology.orchestra.run.smithi162.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_location": "", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_location_hook": "", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_location_hook_timeout": "10", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "daemonize": "false", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_alienstore": "0/5", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_asok": "1/5", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_asok_assert_abort": "false", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T11:13:11.466 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_auth": "20/20", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_bdev": "1/3", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_bluefs": "1/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_bluestore": "1/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_buffer": "0/1", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_cephsqlite": "0/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_civetweb": "1/10", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_client": "0/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_compressor": "1/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_context": "0/1", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_crush": "1/1", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_crypto": "1/5", 2024-09-16T11:13:11.467 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_dpdk": "1/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_eventtrace": "1/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_filer": "0/1", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_filestore": "1/3", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_finisher": "1/1", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_fuse": "1/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_javaclient": "1/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_journal": "1/3", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_journaler": "0/5", 2024-09-16T11:13:11.468 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_kstore": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_leveldb": "4/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_lockdep": "0/1", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mclock": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mds": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mds_balancer": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mds_locker": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mds_log": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mds_migrator": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_memdb": "4/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mgr": "2/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mgrc": "1/5", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_mon": "20/20", 2024-09-16T11:13:11.469 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_monc": "0/10", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_ms": "1/1", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_none": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_objclass": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_objectcacher": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_objecter": "0/1", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_optracker": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_osd": "1/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_paxos": "20/20", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_perfcounter": "1/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_prioritycache": "1/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rados": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rbd": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T11:13:11.470 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rbd_replay": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_refs": "0/0", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_reserver": "1/1", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rgw": "1/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rgw_sync": "1/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_rocksdb": "4/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_cache": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_device": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_journal": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_lba": "0/5", 2024-09-16T11:13:11.471 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_odata": "0/5", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_omap": "0/5", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_onode": "0/5", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_seastore_tm": "0/5", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_striper": "0/1", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_test": "0/5", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_throttle": "1/1", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_timer": "0/1", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "debug_tp": "0/5", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "device_failure_prediction_mode": "none", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "err_to_graylog": "false", 2024-09-16T11:13:11.472 INFO:teuthology.orchestra.run.smithi162.stdout: "err_to_journald": "false", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "err_to_stderr": "true", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "err_to_syslog": "false", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event_tracing": "false", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "exporter_http_port": "9926", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "exporter_prio_limit": "5", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "exporter_sort_metrics": "true", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "exporter_stats_period": "5", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "fake_statfs_for_testing": "0", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "fatal_signal_handlers": "true", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "filer_max_purge_ops": "10", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "filer_max_truncate_ops": "128", 2024-09-16T11:13:11.473 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_blackhole": "false", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_btrfs_snap": "true", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_caller_concurrency": "10", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_debug_omap_check": "false", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_debug_verify_split": "false", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_dump_file": "", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T11:13:11.474 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fadvise": "true", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fail_eio": "true", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fd_cache_size": "128", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fiemap": "false", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_inject_stall": "0", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_journal_parallel": "false", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_journal_trailing": "false", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_journal_writeahead": "false", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_kill_at": "0", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T11:13:11.475 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T11:13:11.476 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_merge_threshold": "-10", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_odsync_write": "false", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_omap_backend_path": "", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_op_threads": "2", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_punch_hole": "false", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T11:13:11.477 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_queue_max_ops": "50", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_seek_data_hole": "false", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_sloppy_crc": "false", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_splice": "false", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_split_multiple": "2", 2024-09-16T11:13:11.478 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_split_rand_factor": "20", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_update_to": "1000", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T11:13:11.479 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_xfs_extsize": "false", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "filestore_zfs_snap": "false", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fio_dir": "/tmp/fio", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7e3ff0c6-8090-4b11-95d6-fd611813de49", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_allow_other": "true", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_big_writes": "true", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_debug": "false", 2024-09-16T11:13:11.480 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_default_permissions": "false", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_disable_pagecache": "false", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_max_write": "0", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_multithreaded": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_require_active_mds": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_set_user_groups": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_splice_move": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_splice_read": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_splice_write": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "gss_target_name": "ceph", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_file": "", 2024-09-16T11:13:11.481 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_inject_failure": "0", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_interval": "5", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "host": "smithi162", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T11:13:11.482 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "inject_early_sigterm": "false", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "jaeger_tracing_enable": "false", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_aio": "true", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_align_min_size": "65536", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_block_align": "true", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_block_size": "4096", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_dio": "true", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_discard": "false", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_force_aio": "false", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_ignore_corruption": "false", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T11:13:11.483 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_max_write_entries": "100", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_replay_from": "0", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_write_header_frequency": "0", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journal_zero_on_create": "false", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journaler_prefetch_periods": "10", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journaler_prezero_periods": "5", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "journaler_write_head_interval": "15", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "key": "", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "keyfile": "", 2024-09-16T11:13:11.484 INFO:teuthology.orchestra.run.smithi162.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_backend": "rocksdb", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_max_bytes": "67108864", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_max_ops": "512", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_onode_map_size": "1024", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "kstore_sync_transaction": "false", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_block_size": "65536", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_bloom_size": "0", 2024-09-16T11:13:11.485 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_cache_size": "536870912", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_compact_on_mount": "false", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_compression": "false", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_log": "", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_max_open_files": "0", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_paranoid": "false", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "librados_thread_count": "2", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "lockdep": "false", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "lockdep_force_backtrace": "false", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "log_coarse_timestamps": "true", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "log_flush_on_exit": "false", 2024-09-16T11:13:11.486 INFO:teuthology.orchestra.run.smithi162.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_graylog_port": "12201", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_max_new": "1000", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_max_recent": "10000", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_stderr_prefix": "", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_to_file": "true", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_to_graylog": "false", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_to_journald": "false", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_to_stderr": "false", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "log_to_syslog": "false", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_action_on_write_error": "1", 2024-09-16T11:13:11.487 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_alternate_name_max": "8192", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_asio_thread_count": "2", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_export_pin": "true", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_interval": "10", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_max": "-1", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_max_until": "-1", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_merge_size": "50", 2024-09-16T11:13:11.488 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_mode": "0", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_split_bits": "3", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_split_size": "10000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T11:13:11.489 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_mid": "0.700000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_trim_interval": "1", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T11:13:11.490 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_debug_auth_pins": "false", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_debug_frag": "false", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_debug_scatterstat": "false", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_debug_subtrees": "false", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_default_dir_hash": "2", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_defer_session_stale": "true", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T11:13:11.491 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dir_max_entries": "0", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_early_reply": "true", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_enable_op_tracker": "true", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_enforce_unique_name": "true", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T11:13:11.492 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_inject_health_dummy": "false", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T11:13:11.493 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_join_fs": "", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_journal_format": "1", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_create_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_export_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_import_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_journal_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_link_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_openc_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_rename_at": "0", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-09-16T11:13:11.494 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_max_events": "-1", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_max_segments": "128", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_pause": "false", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_segment_size": "0", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_completed_requests": "100000", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_export_size": "20971520", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_file_recover": "32", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_purge_files": "64", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_purge_ops": "8192", 2024-09-16T11:13:11.495 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_max_xattr_pairs_size": "65536", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_metrics_update_interval": "2", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_min_caps_per_client": "100", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_numa_node": "-1", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_op_history_duration": "600", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_op_history_size": "20", 2024-09-16T11:13:11.496 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_op_log_threshold": "5", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_ping_grace": "15", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_ping_interval": "5", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_recall_max_caps": "30000", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_replay_interval": "1.000000", 2024-09-16T11:13:11.497 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_root_ino_gid": "0", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_root_ino_uid": "0", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T11:13:11.498 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_shutdown_check": "0", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_skip_ino": "0", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_snap_min_uid": "0", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_snap_rstat": "false", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_symlink_recovery": "true", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_thrash_exports": "0", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_thrash_fragments": "0", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_tick_interval": "5.000000", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_valgrind_exit": "false", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_verify_backtrace": "1", 2024-09-16T11:13:11.499 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_verify_scatter": "false", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_wipe_sessions": "false", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool_debug": "false", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "memstore_page_set": "false", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "memstore_page_size": "65536", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_client_bytes": "134217728", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_client_messages": "512", 2024-09-16T11:13:11.500 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_disabled_modules": "diskprediction_local", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_mds_messages": "128", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_mon_messages": "128", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_osd_messages": "8192", 2024-09-16T11:13:11.501 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_pool": "true", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_standby_modules": "true", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_stats_period": "5", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_stats_threshold": "5", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_tick_period": "2", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_allow_pool_delete": "true", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_bytes": "104857600", 2024-09-16T11:13:11.502 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T11:13:11.503 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_level": "debug", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_compact_on_start": "false", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_compact_on_trim": "true", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T11:13:11.504 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_cpu_threads": "4", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_data_avail_crit": "5", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_data_avail_warn": "5", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_dump_json": "false", 2024-09-16T11:13:11.505 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_extra_checks": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_no_require_pacific": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_no_require_quincy": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_election_default_strategy": "1", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_election_timeout": "5.000000", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T11:13:11.506 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_enable_op_tracker": "true", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_fake_pool_delete": "false", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_force_quorum_join": "false", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_health_log_update_period": "5", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_health_max_detail": "50", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_health_to_clog": "true", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_host": "172.21.15.162,[v2:172.21.15.162:3301,v1:172.21.15.162:6790],172.21.15.165", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_host_override": "", 2024-09-16T11:13:11.507 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_initial_members": "", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_lease": "5.000000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_log_full_interval": "50", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_log_max": "10000", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_log_max_summary": "50", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T11:13:11.508 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_log_epochs": "500", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_osd": "10000", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_memory_autotune": "true", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_memory_target": "2147483648", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T11:13:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mgr_digest_period": "5", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_op_complaint_time": "30", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_op_history_duration": "600", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_op_history_size": "20", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_op_log_threshold": "5", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T11:13:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_cache_size": "500", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T11:13:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T11:13:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_report_timeout": "900", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T11:13:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_reweight_max_osds": "4", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_scrub_interval": "86400", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_scrub_max_keys": "100", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_scrub_timeout": "300", 2024-09-16T11:13:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_session_timeout": "300", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_smart_report_timeout": "5", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_stretch_pool_size": "4", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_sync_debug": "false", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T11:13:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_tick_interval": "5", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T11:13:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_older_version": "true", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "monmap": "", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_op_threads": "3", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T11:13:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_cm": "false", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_device_name": "", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_sl": "3", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T11:13:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_rdma_type": "ib", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_async_reap_threshold": "5", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_before_connect": "false", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_ipv4": "true", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_ipv6": "false", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_msgr1": "true", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_msgr2": "true", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_port_max": "7568", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_port_min": "6800", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_retry_count": "3", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_bind_retry_delay": "5", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_blackhole_client": "false", 2024-09-16T11:13:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_blackhole_mds": "false", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_blackhole_mgr": "false", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_blackhole_mon": "false", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_blackhole_osd": "false", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_client_mode": "crc secure", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_cluster_type": "", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_compress_secure": "false", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_crc_data": "true", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_crc_header": "true", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_die_on_bug": "true", 2024-09-16T11:13:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_die_on_old_message": "true", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_hugepages": "", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_lro": "true", 2024-09-16T11:13:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_pmd": "", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_port_id": "0", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_dump_on_send": "false", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_inject_delay_type": "", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_inject_socket_failures": "5000", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_max_accept_failures": "4", 2024-09-16T11:13:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_max_backoff": "15.000000", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_osd_compress_mode": "none", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_pq_min_cost": "65536", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_public_type": "", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_service_mode": "crc secure", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_tcp_nodelay": "true", 2024-09-16T11:13:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "ms_type": "async+posix", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "no_config_file": "false", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_inflight_ops": "1024", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter_timeout": "10.000000", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "objectstore_blackhole": "false", 2024-09-16T11:13:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "openssl_engine_opts": "", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_max_ops": "4", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_agent_slop": "0.020000", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_asio_thread_count": "2", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T11:13:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_backfill_scan_max": "512", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_backfill_scan_min": "64", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_backoff_on_peering": "false", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_beacon_report_interval": "300", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bench_duration": "30", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_blkin_trace_all": "false", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T11:13:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_class_update_on_start": "true", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_client_message_cap": "256", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_client_op_priority": "63", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_client_watch_timeout": "30", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_command_max_records": "256", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_compact_on_start": "false", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T11:13:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_crush_update_on_start": "true", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T11:13:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_op_order": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_shutdown": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T11:13:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_default_notify_timeout": "30", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_enable_op_tracker": "true", 2024-09-16T11:13:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_fast_info": "true", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_fast_shutdown": "true", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_function_tracing": "false", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_grace": "20", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_interval": "6", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T11:13:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_stale": "600", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_journal_size": "5120", 2024-09-16T11:13:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_kill_backfill_at": "0", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_map_cache_size": "50", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_map_dedup": "true", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_map_message_max": "40", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_attr_name_len": "100", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_attr_size": "0", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_backfills": "1", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_markdown_count": "5", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_markdown_period": "600", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_object_name_len": "2048", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_object_size": "134217728", 2024-09-16T11:13:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_pgls": "1024", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_push_cost": "8388608", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_push_objects": "10", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_scrubs": "3", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_max_write_size": "90", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T11:13:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_profile": "balanced", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T11:13:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_base": "805306368", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_target": "4294967296", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_target_autotune": "false", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T11:13:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_min_recovery_priority": "0", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mon_report_interval": "5", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_num_cache_shards": "32", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_numa_node": "-1", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_objecter_finishers": "1", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_objectstore": "bluestore", 2024-09-16T11:13:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_objectstore_fuse": "false", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_objectstore_tracing": "false", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_history_duration": "600", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_history_size": "20", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_log_threshold": "5", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_num_shards": "0", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T11:13:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_op_thread_timeout": "15", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_open_classes_on_start": "true", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_os_flags": "0", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_peering_op_priority": "255", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_delete_priority": "5", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T11:13:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T11:13:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_flags": "0", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_min_size": "0", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_size": "2", 2024-09-16T11:13:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_default_type": "replicated", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_cost": "20971520", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_max_active": "0", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T11:13:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_op_priority": "3", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_priority": "5", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_repair_during_recovery": "false", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T11:13:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_cost": "52428800", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_end_hour": "0", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T11:13:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_priority": "5", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_skip_data_digest": "false", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_smart_report_timeout": "5", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T11:13:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_snap_trim_priority": "5", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_target_transaction_size": "30", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T11:13:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_tracing": "false", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_use_stale_snap": "false", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_kill_at": "0", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_max_join_drift": "10", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_min": "500", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_min_wait": "0.050000", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_service_trim_max": "500", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_service_trim_min": "250", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_stash_full_interval": "25", 2024-09-16T11:13:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_trim_max": "500", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "paxos_trim_min": "250", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "perf": "true", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "-", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrv": "", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "public_bind_addr": "-", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "public_network": "", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "public_network_interface": "", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "qat_compressor_enabled": "false", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "rados_mon_op_timeout": "0", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "rados_osd_op_timeout": "0", 2024-09-16T11:13:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "rados_tracing": "false", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_atime_update_interval": "60", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache": "true", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_policy": "writearound", 2024-09-16T11:13:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_size": "33554432", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_compression_hint": "none", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_clone_format": "auto", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_data_pool": "", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_features": "61", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_format": "2", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_map_options": "", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_order": "22", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_pool": "rbd", 2024-09-16T11:13:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_stripe_count": "0", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_io_scheduler": "simple", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T11:13:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_order": "24", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_pool": "", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_journal_splay_width": "4", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T11:13:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T11:13:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T11:13:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_op_threads": "1", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_plugins": "", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T11:13:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T11:13:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_tracing": "false", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_validate_names": "true", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd_validate_pool": "true", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "restapi_base_url": "", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "restapi_log_level": "", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T11:13:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_admin_entry": "admin", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_backend_store": "rados", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_barbican_url": "", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_beast_enable_async": "true", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_cache_enabled": "true", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_cache_lru_size": "10000", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_content_length_compat": "false", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T11:13:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_cross_domain_policy": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T11:13:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T11:13:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T11:13:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_data_log_window": "30", 2024-09-16T11:13:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_data_notify_interval_msec": "200", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T11:13:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dns_name": "", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dns_s3website_name": "", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T11:13:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_ops_log": "false", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_static_website": "false", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enable_usage_log": "false", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_expose_bucket": "false", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_extended_http_attrs": "", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T11:13:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_max_objs": "32", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T11:13:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_init_timeout": "300", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_password": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_project": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T11:13:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_token": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_admin_user": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_api_version": "2", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T11:13:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_url": "", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_max_objs": "32", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_max_worker": "3", 2024-09-16T11:13:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T11:13:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T11:13:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T11:13:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ldap_uri": "", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_log_http_headers": "", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T11:13:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_attr_size": "0", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_listing_results": "1000", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_notify_retries": "10", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T11:13:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T11:13:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_num_control_oids": "8", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_numa_node": "-1", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T11:13:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_op_tracing": "false", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_opa_token": "", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_opa_url": "", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ops_log_file_path": "", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ops_log_rados": "true", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_print_continue": "true", 2024-09-16T11:13:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_rados_tracing": "false", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_realm": "", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_realm_id": "", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_region": "", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T11:13:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_request_uri": "", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_resolve_cname": "false", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_run_sync_thread": "true", 2024-09-16T11:13:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_script_uri": "", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_service_provider_name": "", 2024-09-16T11:13:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_client_id": "", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_client_secret": "", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_entry": "sts", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_key": "sts", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_auth_url": "", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_custom_header": "", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_need_stats": "true", 2024-09-16T11:13:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_tenant_name": "", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_url": "", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_lease_period": "120", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T11:13:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_thread_pool_size": "512", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_comment": "", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_createby": "", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_encoding": "", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_flag": "false", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_origin": "", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_torrent_tracker": "", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_usage_max_shards": "32", 2024-09-16T11:13:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_use_opa_authz": "false", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_user_unique_email": "true", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_verify_ssl": "true", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_zone": "", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_zone_id": "", 2024-09-16T11:13:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_zonegroup": "", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_zonegroup_id": "", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_block_size": "4096", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T11:13:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_partition_filters": "false", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_perf": "false", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "run_dir": "/var/run/ceph", 2024-09-16T11:13:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_block_create": "true", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_device_size": "107374182400", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "seastore_segment_size": "67108864", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "setgroup": "", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "setuser": "", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "setuser_match_path": "", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T11:13:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "thp": "false", 2024-09-16T11:13:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "threadpool_default_timeout": "60", 2024-09-16T11:13:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T11:13:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "throttler_perf_counter": "true" 2024-09-16T11:13:11.588 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:11.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T11:13:11.722 INFO:teuthology.orchestra.run.smithi162.stdout:mon.a: {} 2024-09-16T11:13:11.737 INFO:teuthology.orchestra.run.smithi162.stdout:mon.b: {} 2024-09-16T11:13:11.753 INFO:teuthology.orchestra.run.smithi162.stdout:mon.c: {} 2024-09-16T11:13:11.756 INFO:teuthology.orchestra.run.smithi162.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:13:11.756 INFO:teuthology.orchestra.run.smithi162.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:13:11.756 INFO:teuthology.orchestra.run.smithi162.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:13:11.767 DEBUG:teuthology.orchestra.run.smithi162:> type ceph-objectstore-tool 2024-09-16T11:13:11.773 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T11:13:11.774 DEBUG:teuthology.orchestra.run.smithi165:> type ceph-objectstore-tool 2024-09-16T11:13:11.780 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T11:13:11.780 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T11:13:11.790 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T11:13:11.790 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T11:13:11.790 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:11.790 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:11.790 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T11:13:11.790 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:13:11.791 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T11:13:11.791 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T11:13:11.791 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:11.792 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T11:13:11.792 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:11.792 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T11:13:11.792 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:13:11.793 DEBUG:teuthology.orchestra.run.smithi162:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-16T11:13:11.794 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:11.785+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:11.816 INFO:teuthology.orchestra.run.smithi162.stdout:mapped writes DISABLED 2024-09-16T11:13:11.816 INFO:teuthology.orchestra.run.smithi162.stdout:mapped reads DISABLED 2024-09-16T11:13:11.816 INFO:teuthology.orchestra.run.smithi162.stdout:Seed set to 9221 2024-09-16T11:13:11.894 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:11.888+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:11.897 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:11.897 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:11.898 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:11.898 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:11.909 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:11.951 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:11.973 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:11.990 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:11.993 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:11.989+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:12.004 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.004 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.007 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:12.014 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:12.031 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:12.053 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:12.087 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:12.093 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:12.089+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:12.114 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:12.115 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:12.116 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:12.125 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.125 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.128 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 31, 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7e3ff0c6-8090-4b11-95d6-fd611813de49", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2024-09-16T11:12:35.459871+0000", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2024-09-16T11:13:12.078339+0000", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2024-09-16T11:12:46.645944+0000", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2024-09-16T11:12:37.534334+0000", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5799936, 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2024-09-16T11:13:12.129 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 4, 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "quincy", 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2024-09-16T11:13:12.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": ".mgr", 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:48.004741+0000", 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:12.131 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "17", 2024-09-16T11:13:12.132 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:12.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:12.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2024-09-16T11:13:12.135 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_max": 32, 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": {} 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:53.361644+0000", 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:12.136 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2024-09-16T11:13:12.137 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "26", 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:12.138 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [ 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: 3 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": 3, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": 3, 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:12.139 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:12.140 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 3, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cache", 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:02.524586+0000", 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8201, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:12.142 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 4, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 4, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 4, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 4, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 4, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:12.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "30", 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:12.144 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": 2, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "writeback", 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 250, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "bloom", 2024-09-16T11:13:12.145 INFO:teuthology.orchestra.run.smithi162.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "target_size": 0, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "seed": 0 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 60, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 8, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [ 2024-09-16T11:13:12.146 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:12.147 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "pool_client.0", 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:11.876887+0000", 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 32769, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,creating", 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:12.148 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 32, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 32, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 32, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 32, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 32, 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:12.149 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "31", 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:12.150 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:12.151 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:12.152 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": {} 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b772f030-6c52-46c6-a992-36cadd7f9edd", 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.153 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 23, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6825/22248", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6827/22248", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6831/22248", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6829/22248", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6335aa8f-a82b-45a4-be04-b1b064314f8d", 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 23, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6800", 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6801", 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2024-09-16T11:13:12.160 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2024-09-16T11:13:12.161 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6801/22247", 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6803/22247", 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6807/22247", 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6805/22247", 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.163 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "8d9432da-1713-44e1-9c96-db33f21cf333", 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 18, 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2024-09-16T11:13:12.165 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6817/22258", 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6819/22258", 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6823/22258", 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6821/22258", 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "f17d5292-6b43-4887-a146-a647c60e4f86", 2024-09-16T11:13:12.169 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 23, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.170 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.171 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2024-09-16T11:13:12.172 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.173 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6809/22257", 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6811/22257", 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6815/22257", 2024-09-16T11:13:12.174 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6813/22257", 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "93cb4050-ba1d-44c4-8e05-854a4cbb9236", 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.175 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 23, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6800", 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6801", 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6802", 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6803", 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6806", 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6807", 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6804", 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.179 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6805", 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6801/22379", 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6803/22379", 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6807/22379", 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6805/22379", 2024-09-16T11:13:12.180 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "432a0446-4f3b-436d-bba0-76184260e13c", 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.181 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 18, 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6808", 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6809", 2024-09-16T11:13:12.182 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6810", 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.183 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6811", 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6814", 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.184 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6815", 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6812", 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.185 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6813", 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6809/22380", 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6811/22380", 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6815/22380", 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6813/22380", 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:13:12.186 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "7c7e65e2-ab06-47f4-8d6d-712aa308e13f", 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 18, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.187 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6816", 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6818", 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.188 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6819", 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6821", 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6826", 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6827", 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.190 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6822", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6824", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6818/22381", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6821/22381", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6827/22381", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6824/22381", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.191 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "827daf60-a01e-40a5-9595-7ec5406e4a8a", 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 18, 2024-09-16T11:13:12.192 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6817", 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6820", 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6823", 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6825", 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.194 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6830", 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6831", 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6828", 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6829", 2024-09-16T11:13:12.196 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6820/22378", 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6825/22378", 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6831/22378", 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6829/22378", 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2024-09-16T11:13:12.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.054827+0000", 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.198 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.474331+0000", 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.753279+0000", 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.733978+0000", 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:13:12.200 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.980000+0000", 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.201 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.142568+0000", 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.883591+0000", 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.129808+0000", 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2024-09-16T11:13:12.203 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklist": {}, 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "range_blocklist": {}, 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2024-09-16T11:13:12.204 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {}, 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode": { 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_enabled": false, 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_bucket_count": 0, 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_bucket": 0 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:12.205 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:12.207 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:12.208 INFO:tasks.thrashosds.thrasher:j is {'epoch': 31, 'fsid': '7e3ff0c6-8090-4b11-95d6-fd611813de49', 'created': '2024-09-16T11:12:35.459871+0000', 'modified': '2024-09-16T11:13:12.078339+0000', 'last_up_change': '2024-09-16T11:12:46.645944+0000', 'last_in_change': '2024-09-16T11:12:37.534334+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:12:48.004741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:12:53.361644+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:13:02.524586+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:13:11.876887+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}], 'osds': [{'osd': 0, 'uuid': 'b772f030-6c52-46c6-a992-36cadd7f9edd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6825', 'nonce': 22248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6826', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 22248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6830', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 22248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6829', 'nonce': 22248}]}, 'public_addr': '172.21.15.162:6825/22248', 'cluster_addr': '172.21.15.162:6827/22248', 'heartbeat_back_addr': '172.21.15.162:6831/22248', 'heartbeat_front_addr': '172.21.15.162:6829/22248', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6335aa8f-a82b-45a4-be04-b1b064314f8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 22247}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 22247}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 22247}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 22247}]}, 'public_addr': '172.21.15.162:6801/22247', 'cluster_addr': '172.21.15.162:6803/22247', 'heartbeat_back_addr': '172.21.15.162:6807/22247', 'heartbeat_front_addr': '172.21.15.162:6805/22247', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8d9432da-1713-44e1-9c96-db33f21cf333', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6817', 'nonce': 22258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6818', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 22258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6822', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 22258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6821', 'nonce': 22258}]}, 'public_addr': '172.21.15.162:6817/22258', 'cluster_addr': '172.21.15.162:6819/22258', 'heartbeat_back_addr': '172.21.15.162:6823/22258', 'heartbeat_front_addr': '172.21.15.162:6821/22258', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f17d5292-6b43-4887-a146-a647c60e4f86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 22257}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 22257}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 22257}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 22257}]}, 'public_addr': '172.21.15.162:6809/22257', 'cluster_addr': '172.21.15.162:6811/22257', 'heartbeat_back_addr': '172.21.15.162:6815/22257', 'heartbeat_front_addr': '172.21.15.162:6813/22257', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '93cb4050-ba1d-44c4-8e05-854a4cbb9236', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 22379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 22379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 22379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 22379}]}, 'public_addr': '172.21.15.165:6801/22379', 'cluster_addr': '172.21.15.165:6803/22379', 'heartbeat_back_addr': '172.21.15.165:6807/22379', 'heartbeat_front_addr': '172.21.15.165:6805/22379', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '432a0446-4f3b-436d-bba0-76184260e13c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 22380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 22380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 22380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 22380}]}, 'public_addr': '172.21.15.165:6809/22380', 'cluster_addr': '172.21.15.165:6811/22380', 'heartbeat_back_addr': '172.21.15.165:6815/22380', 'heartbeat_front_addr': '172.21.15.165:6813/22380', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c7e65e2-ab06-47f4-8d6d-712aa308e13f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6818', 'nonce': 22381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6819', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 22381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 22381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6824', 'nonce': 22381}]}, 'public_addr': '172.21.15.165:6818/22381', 'cluster_addr': '172.21.15.165:6821/22381', 'heartbeat_back_addr': '172.21.15.165:6827/22381', 'heartbeat_front_addr': '172.21.15.165:6824/22381', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '827daf60-a01e-40a5-9595-7ec5406e4a8a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6817', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6820', 'nonce': 22378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6823', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 22378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 22378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 22378}]}, 'public_addr': '172.21.15.165:6820/22378', 'cluster_addr': '172.21.15.165:6825/22378', 'heartbeat_back_addr': '172.21.15.165:6831/22378', 'heartbeat_front_addr': '172.21.15.165:6829/22378', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.054827+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.474331+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.753279+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.733978+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.980000+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.142568+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.883591+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.129808+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:13:12.208 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T11:13:12.209 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:12.200+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:12.308 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:12.304+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:12.360 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.360 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.369 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:12.410 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:12.405+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:12.465 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.465 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.474 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:12.510 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:12.504+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:12.570 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.570 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:12.625 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:12.621+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:12.676 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.676 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.685 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:12.711 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:12.705+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:12.784 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.785 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.796 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:12.812 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:12.808+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:12.896 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:12.896 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:12.905 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:12.913 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:12.909+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:13.012 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.012 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.013 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:13.009+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:13.021 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:13.112 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:13:13.114 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:13.109+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:13.175 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.175 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.184 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:13.215 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:13.208+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:13.280 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.280 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.290 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:13.337 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:13.332+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:13.401 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.402 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.411 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:13.416 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:13.412+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:13.510 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.510 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.517 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:13.512+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:13.519 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:13.618 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:13.612+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:13.663 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.663 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.672 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:13.719 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:13.713+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:13.768 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.768 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.777 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:13.821 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:13.816+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:13.872 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.873 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:13.922 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:13.916+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:13.976 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:13.976 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:13.985 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:14.027 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:14.020+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:14.088 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:14.088 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:14.097 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:14.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:14.140 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:14.136+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:14.229 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:14.224+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:14.243 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:14.244 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:14.253 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:14.272 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:14.293 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:14.314 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:14.330 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:14.325+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:14.335 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:14.352 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:14.359 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:14.359 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:14.369 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:14.375 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:14.394 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:14.415 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:14.417 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:14.417 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:14.418 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:14.418 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:14.418 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:14.418 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:14.418 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:14.418 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:14.430 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:14.424+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:14.470 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:14.470 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:14.470 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:14.470 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:14.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:14.531 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:14.524+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:14.624 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:14.625 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:14.625 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:14.625 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:14.625 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:14.631 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:14.624+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:14.633 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:14.732 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:14.728+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:14.785 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:14.785 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.002141364, 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.786 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:14.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:14.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:14.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:14.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:14.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:14.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:14.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:14.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:14.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:14.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:14.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:14.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:14.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:14.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:14.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:14.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:14.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:14.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:14.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:14.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.798 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.799 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.799 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.799 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:14.799 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.614215664, 2024-09-16T11:13:14.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:14.800 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.800 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.800 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.800 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.801 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.801 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:14.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.801 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:14.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:14.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:14.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:14.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:14.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:14.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:14.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:14.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:14.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:14.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:14.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:14.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:14.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:14.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:14.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:14.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:14.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:14.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:14.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:14.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:14.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:14.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.813 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.814 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.814 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:14.815 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.614064333999998, 2024-09-16T11:13:14.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:14.815 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.815 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.815 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.816 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.816 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.816 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:14.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.816 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:14.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:14.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:14.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:14.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:14.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:14.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:14.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:14.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.614012848000002, 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.826 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:14.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:14.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:14.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:14.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:14.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:14.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:14.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:14.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:14.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:14.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:14.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.834 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:14.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.613942699999999, 2024-09-16T11:13:14.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:14.836 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.836 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.836 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:14.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.837 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:14.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:14.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:14.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:14.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:14.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:14.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:14.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:14.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:14.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:14.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:14.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:14.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:14.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:14.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:14.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:14.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:14.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:14.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:14.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:14.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:14.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:14.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.852 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:14.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.61377023, 2024-09-16T11:13:14.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:14.854 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.854 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.855 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:14.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.855 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:14.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:14.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:14.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.613711907999999, 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.862 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:14.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.867 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.613559788, 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:14.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:14.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.873 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.613381269000001, 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:14.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:14.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:14.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.613348296000002, 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.880 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:14.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.061182804000001, 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.886 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:14.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:14.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.061160570999998, 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.892 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:14.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:14.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.061031774, 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:14.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.060973429000001, 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:14.904 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:14.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:14.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.060919040000002, 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:14.910 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:14.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.060784732999998, 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:14.916 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:14.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.279687212999999, 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:14.922 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:14.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:14.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.279629364000002, 2024-09-16T11:13:14.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:14.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:14.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.279542052, 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:14.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.940 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.279303924000001, 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:14.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:14.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:14.946 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:14.948 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:14.944+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:14.948 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:15.049 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:15.044+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:15.057 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:15.057 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.067 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:15.125 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:13:15.150 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:15.144+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:15.170 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:15.171 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.180 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:15.251 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:15.245+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:15.281 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.281 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:15.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:15.281 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:15.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.282 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:25 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'15, mlcod=16'11)", 2024-09-16T11:13:15.282 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:49.961379+0000", 2024-09-16T11:13:15.282 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.315114852000001, 2024-09-16T11:13:15.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031483069999999999, 2024-09-16T11:13:15.283 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.283 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961379+0000", 2024-09-16T11:13:15.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961379+0000", 2024-09-16T11:13:15.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961380+0000", 2024-09-16T11:13:15.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.066e-06 2024-09-16T11:13:15.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961390+0000", 2024-09-16T11:13:15.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0352e-05 2024-09-16T11:13:15.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961409+0000", 2024-09-16T11:13:15.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8425999999999999e-05 2024-09-16T11:13:15.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961417+0000", 2024-09-16T11:13:15.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5930000000000006e-06 2024-09-16T11:13:15.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961499+0000", 2024-09-16T11:13:15.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1460999999999998e-05 2024-09-16T11:13:15.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.961544+0000", 2024-09-16T11:13:15.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4693000000000001e-05 2024-09-16T11:13:15.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.964442+0000", 2024-09-16T11:13:15.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028982209999999999 2024-09-16T11:13:15.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:49.964527+0000", 2024-09-16T11:13:15.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5494999999999994e-05 2024-09-16T11:13:15.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:39 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'19)", 2024-09-16T11:13:15.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.166991+0000", 2024-09-16T11:13:15.293 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.109502405000001, 2024-09-16T11:13:15.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026177150000000001, 2024-09-16T11:13:15.293 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.294 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.166991+0000", 2024-09-16T11:13:15.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.166991+0000", 2024-09-16T11:13:15.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.166993+0000", 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.142e-06 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.167018+0000", 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5080999999999999e-05 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.167035+0000", 2024-09-16T11:13:15.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7696000000000001e-05 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.167049+0000", 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3532e-05 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.167128+0000", 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9065000000000003e-05 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.167153+0000", 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4912000000000001e-05 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.169503+0000", 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023499369999999999 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.169609+0000", 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010635 2024-09-16T11:13:15.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:40 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'22, mlcod=16'19)", 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.168358+0000", 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.108136026, 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002930607, 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168358+0000", 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168358+0000", 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168359+0000", 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3900000000000003e-07 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168367+0000", 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4749999999999993e-06 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168397+0000", 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0207999999999999e-05 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168407+0000", 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4169999999999998e-06 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168550+0000", 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143514 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.168635+0000", 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4983999999999998e-05 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171224+0000", 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025886020000000002 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171288+0000", 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4467999999999993e-05 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:41 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'23, mlcod=16'19)", 2024-09-16T11:13:15.305 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.169946+0000", 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.10654778, 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002475784, 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.169946+0000", 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.169946+0000", 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.169947+0000", 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1999999999999999e-07 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.170029+0000", 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2508000000000004e-05 2024-09-16T11:13:15.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.170044+0000", 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5241e-05 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.170053+0000", 2024-09-16T11:13:15.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0730000000000005e-06 2024-09-16T11:13:15.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.170143+0000", 2024-09-16T11:13:15.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0200999999999999e-05 2024-09-16T11:13:15.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.170182+0000", 2024-09-16T11:13:15.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9079000000000003e-05 2024-09-16T11:13:15.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.172280+0000", 2024-09-16T11:13:15.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020982710000000001 2024-09-16T11:13:15.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.172422+0000", 2024-09-16T11:13:15.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141691 2024-09-16T11:13:15.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.312 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.313 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:42 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'24, mlcod=16'19)", 2024-09-16T11:13:15.313 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.171628+0000", 2024-09-16T11:13:15.313 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.104866121000001, 2024-09-16T11:13:15.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035115849999999998, 2024-09-16T11:13:15.314 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.314 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171628+0000", 2024-09-16T11:13:15.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171628+0000", 2024-09-16T11:13:15.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171629+0000", 2024-09-16T11:13:15.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.71e-07 2024-09-16T11:13:15.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171636+0000", 2024-09-16T11:13:15.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3239999999999997e-06 2024-09-16T11:13:15.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171654+0000", 2024-09-16T11:13:15.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8546000000000002e-05 2024-09-16T11:13:15.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171662+0000", 2024-09-16T11:13:15.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4019999999999996e-06 2024-09-16T11:13:15.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171736+0000", 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3730000000000001e-05 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.171764+0000", 2024-09-16T11:13:15.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8259000000000002e-05 2024-09-16T11:13:15.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175046+0000", 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003282025 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175139+0000", 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3428000000000001e-05 2024-09-16T11:13:15.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.323 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:43 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'25, mlcod=16'21)", 2024-09-16T11:13:15.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.172982+0000", 2024-09-16T11:13:15.324 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.103511727000001, 2024-09-16T11:13:15.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034968849999999999, 2024-09-16T11:13:15.324 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.324 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.324 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.172982+0000", 2024-09-16T11:13:15.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.172982+0000", 2024-09-16T11:13:15.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.172983+0000", 2024-09-16T11:13:15.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3200000000000001e-06 2024-09-16T11:13:15.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.172993+0000", 2024-09-16T11:13:15.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.842e-06 2024-09-16T11:13:15.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173018+0000", 2024-09-16T11:13:15.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4445000000000001e-05 2024-09-16T11:13:15.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173026+0000", 2024-09-16T11:13:15.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5369999999999997e-06 2024-09-16T11:13:15.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173131+0000", 2024-09-16T11:13:15.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010492699999999999 2024-09-16T11:13:15.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173166+0000", 2024-09-16T11:13:15.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.485e-05 2024-09-16T11:13:15.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176405+0000", 2024-09-16T11:13:15.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032387760000000001 2024-09-16T11:13:15.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176479+0000", 2024-09-16T11:13:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4188000000000002e-05 2024-09-16T11:13:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.334 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:44 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'22)", 2024-09-16T11:13:15.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.173890+0000", 2024-09-16T11:13:15.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.102603631000001, 2024-09-16T11:13:15.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048216300000000004, 2024-09-16T11:13:15.335 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.335 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.335 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173890+0000", 2024-09-16T11:13:15.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173890+0000", 2024-09-16T11:13:15.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173891+0000", 2024-09-16T11:13:15.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.232e-06 2024-09-16T11:13:15.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173973+0000", 2024-09-16T11:13:15.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1514999999999995e-05 2024-09-16T11:13:15.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173993+0000", 2024-09-16T11:13:15.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9677000000000002e-05 2024-09-16T11:13:15.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174001+0000", 2024-09-16T11:13:15.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2309999999999996e-06 2024-09-16T11:13:15.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174084+0000", 2024-09-16T11:13:15.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3512999999999998e-05 2024-09-16T11:13:15.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174118+0000", 2024-09-16T11:13:15.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3312999999999998e-05 2024-09-16T11:13:15.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.178647+0000", 2024-09-16T11:13:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045288799999999999 2024-09-16T11:13:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.178712+0000", 2024-09-16T11:13:15.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5269000000000001e-05 2024-09-16T11:13:15.344 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.344 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.344 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.345 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:45 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'27, mlcod=16'22)", 2024-09-16T11:13:15.345 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.174640+0000", 2024-09-16T11:13:15.345 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.101854261, 2024-09-16T11:13:15.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044158899999999996, 2024-09-16T11:13:15.346 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.346 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174640+0000", 2024-09-16T11:13:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174640+0000", 2024-09-16T11:13:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174641+0000", 2024-09-16T11:13:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.133e-06 2024-09-16T11:13:15.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174652+0000", 2024-09-16T11:13:15.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1197e-05 2024-09-16T11:13:15.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174671+0000", 2024-09-16T11:13:15.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9434999999999999e-05 2024-09-16T11:13:15.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174682+0000", 2024-09-16T11:13:15.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1197e-05 2024-09-16T11:13:15.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175186+0000", 2024-09-16T11:13:15.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050357199999999998 2024-09-16T11:13:15.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175214+0000", 2024-09-16T11:13:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8200000000000001e-05 2024-09-16T11:13:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179004+0000", 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037893470000000002 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179055+0000", 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1808999999999998e-05 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:46 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'28, mlcod=16'22)", 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.175230+0000", 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.101264283999999, 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048366809999999998, 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175230+0000", 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175230+0000", 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175231+0000", 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0470000000000001e-06 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175240+0000", 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4110000000000002e-06 2024-09-16T11:13:15.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175254+0000", 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4064e-05 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175267+0000", 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2526e-05 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175839+0000", 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057268499999999997 2024-09-16T11:13:15.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175878+0000", 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8671999999999998e-05 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180028+0000", 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041503729999999997 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180066+0000", 2024-09-16T11:13:15.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7902999999999999e-05 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:47 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'29, mlcod=16'22)", 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.175934+0000", 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.100559337, 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041741260000000002, 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175934+0000", 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175934+0000", 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175936+0000", 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2559999999999999e-06 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176009+0000", 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3695000000000005e-05 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176023+0000", 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3551e-05 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176044+0000", 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1162000000000002e-05 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176544+0000", 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049992299999999999 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176576+0000", 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1949999999999997e-05 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180074+0000", 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034984479999999999 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180109+0000", 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4140999999999997e-05 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:48 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'30, mlcod=16'24)", 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.177471+0000", 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.099023219999999, 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002676674, 2024-09-16T11:13:15.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177471+0000", 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177471+0000", 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177472+0000", 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2419999999999999e-06 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177481+0000", 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5550000000000008e-06 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177505+0000", 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3898000000000001e-05 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177514+0000", 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4880000000000007e-06 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177600+0000", 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5758000000000001e-05 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.177634+0000", 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4678999999999998e-05 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180115+0000", 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002480866 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180147+0000", 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2187999999999997e-05 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.367 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:51 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'24)", 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.179765+0000", 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.096728611, 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003872014, 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179765+0000", 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179765+0000", 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179766+0000", 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179772+0000", 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4459999999999998e-06 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179793+0000", 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0095e-05 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179800+0000", 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2590000000000003e-06 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180220+0000", 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000419875 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180278+0000", 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8659000000000002e-05 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.183591+0000", 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033125870000000001 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.183637+0000", 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6226000000000001e-05 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.371 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:52 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'34, mlcod=16'30)", 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.181419+0000", 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.095075060999999, 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002749512, 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181419+0000", 2024-09-16T11:13:15.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181419+0000", 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181420+0000", 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2999999999999999e-07 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181426+0000", 2024-09-16T11:13:15.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3960000000000003e-06 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181441+0000", 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4715000000000001e-05 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181459+0000", 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8057e-05 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181508+0000", 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9364999999999999e-05 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.181524+0000", 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5885000000000001e-05 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.184130+0000", 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026056339999999999 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.184168+0000", 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8529999999999999e-05 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:55 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'37, mlcod=16'33)", 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.187537+0000", 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.088956559, 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028297130000000002, 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187537+0000", 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187537+0000", 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187538+0000", 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0139999999999999e-06 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187637+0000", 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8831000000000003e-05 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187687+0000", 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9885000000000001e-05 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187695+0000", 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2379999999999997e-06 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187734+0000", 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8423999999999997e-05 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.187758+0000", 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4241000000000001e-05 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.190327+0000", 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025688260000000002 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.190367+0000", 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0253999999999998e-05 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:65 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'43, mlcod=16'40)", 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.721245+0000", 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.555248445, 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031763749999999999, 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721245+0000", 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721245+0000", 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721247+0000", 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7740000000000001e-06 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721260+0000", 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721288+0000", 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7849000000000001e-05 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721308+0000", 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0276e-05 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721468+0000", 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000159414 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.721503+0000", 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199e-05 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724303+0000", 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028006989999999998 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724422+0000", 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118268 2024-09-16T11:13:15.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:66 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'44, mlcod=16'40)", 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.722122+0000", 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.554371774, 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002647029, 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722122+0000", 2024-09-16T11:13:15.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722122+0000", 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722124+0000", 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7230000000000001e-06 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722333+0000", 2024-09-16T11:13:15.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000209559 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722356+0000", 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2476e-05 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722370+0000", 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3736e-05 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722447+0000", 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7012000000000001e-05 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.722475+0000", 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7996000000000001e-05 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724664+0000", 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021898460000000001 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724769+0000", 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104681 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:67 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'45, mlcod=16'40)", 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.723380+0000", 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.553113594999999, 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025414550000000002, 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723380+0000", 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723380+0000", 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723381+0000", 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.283e-06 2024-09-16T11:13:15.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723397+0000", 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5116999999999999e-05 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723432+0000", 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5299000000000002e-05 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723460+0000", 2024-09-16T11:13:15.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7886e-05 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723619+0000", 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015934 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.723657+0000", 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8052000000000001e-05 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725795+0000", 2024-09-16T11:13:15.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021374279999999998 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725922+0000", 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012705 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'46, mlcod=16'41)", 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.724857+0000", 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.551637067000001, 2024-09-16T11:13:15.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026973349999999999, 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724857+0000", 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724857+0000", 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724858+0000", 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.57e-06 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724869+0000", 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1039999999999999e-05 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724902+0000", 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2301999999999997e-05 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.724917+0000", 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5160000000000001e-05 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725011+0000", 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4158000000000002e-05 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725056+0000", 2024-09-16T11:13:15.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5243e-05 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.727486+0000", 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024296370000000001 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.727554+0000", 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8225000000000005e-05 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'47, mlcod=16'41)", 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.725683+0000", 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.550810994999999, 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039048329999999999, 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725683+0000", 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725683+0000", 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725684+0000", 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3570000000000001e-06 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725770+0000", 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6034999999999999e-05 2024-09-16T11:13:15.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725808+0000", 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7863e-05 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725822+0000", 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4049000000000001e-05 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.726047+0000", 2024-09-16T11:13:15.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022463499999999999 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.726079+0000", 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2703000000000002e-05 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.729518+0000", 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003439059 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.729588+0000", 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9132000000000002e-05 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:71 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'49, mlcod=16'44)", 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.728707+0000", 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.54778683, 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025305290000000001, 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728707+0000", 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728707+0000", 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728709+0000", 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.593e-06 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728721+0000", 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2294e-05 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728752+0000", 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1449000000000003e-05 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728763+0000", 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0995999999999999e-05 2024-09-16T11:13:15.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728871+0000", 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107908 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4755999999999997e-05 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.731131+0000", 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002225089 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.731237+0000", 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106444 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.406 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.408 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:15.408 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:15.404+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:15.504 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:15.504 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.509 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:15.505+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:15.513 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:15.610 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:15.604+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:15.656 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:15.657 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:15.657 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.665 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:15.711 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:15.704+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:55.835591+0000", 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.925076183000002, 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014186299999999999, 2024-09-16T11:13:15.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835591+0000", 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835591+0000", 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835591+0000", 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835599+0000", 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7579999999999992e-06 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835620+0000", 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1590000000000002e-05 2024-09-16T11:13:15.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835630+0000", 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0171e-05 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835767+0000", 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000136372 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.835807+0000", 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0559000000000002e-05 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.836952+0000", 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011443930000000001 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:55.837009+0000", 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7383999999999998e-05 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.766 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:56.842530+0000", 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.918136854, 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013182879999999999, 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842530+0000", 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842530+0000", 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842530+0000", 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842537+0000", 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7909999999999999e-06 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842561+0000", 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3481000000000001e-05 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842574+0000", 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3288000000000001e-05 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842651+0000", 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7591999999999998e-05 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.842709+0000", 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7982999999999997e-05 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.843779+0000", 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010694890000000001 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:56.843848+0000", 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9282000000000005e-05 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:10 4.1c e34/31 4:3831950c:::rbd_directory:head v 34'1, mlcod=0'0)", 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.128069+0000", 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.63259814199999997, 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011218529999999999, 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:15.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128069+0000", 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128069+0000", 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128069+0000", 2024-09-16T11:13:15.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2099999999999997e-07 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128079+0000", 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4250000000000004e-06 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128097+0000", 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8332999999999999e-05 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128108+0000", 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1325000000000001e-05 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128194+0000", 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5721000000000001e-05 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.128230+0000", 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6205000000000003e-05 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129072+0000", 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00084193699999999996 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129190+0000", 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118386 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:15.775 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:15.812 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:15.808+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:15.860 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:15.860 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:15.861 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.869 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:15.912 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:15.905+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:15.959 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:15.959 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:15.967 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:16.014 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:16.008+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:16.078 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 21.246512708000001, 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:16.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:16.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.95593793800000004, 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:16.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:16.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:16.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:16.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:16.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.94752965, 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:16.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.093 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.094 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:16.095 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:16.114 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:16.109+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:16.129 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:13:16.189 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:16.189 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:16.189 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:16.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:16.216 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:16.209+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:16.293 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:16.293 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:16.293 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:16.293 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:16.293 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:16.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:16.345 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:16.341+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:16.418 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:16.412+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:16.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:16.440 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:9 4.1a e34/31 4:5cea7035:::rbd_info:head v 34'2, mlcod=0'0)", 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.115686+0000", 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.323483108, 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022638939999999998, 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115686+0000", 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115686+0000", 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115687+0000", 2024-09-16T11:13:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115695+0000", 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6639999999999998e-06 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115720+0000", 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5182999999999999e-05 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115729+0000", 2024-09-16T11:13:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5980000000000003e-06 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115852+0000", 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122951 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115893+0000", 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0744999999999997e-05 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.117898+0000", 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020051689999999998 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.117950+0000", 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2120000000000002e-05 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13 4.11 4:8e779da3:::rbd_object_map.11b5175acf49:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.142847+0000", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.2963227509999999, 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003404327, 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142847+0000", 2024-09-16T11:13:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142847+0000", 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142849+0000", 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.317e-06 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142859+0000", 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5389999999999996e-06 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142877+0000", 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8652999999999999e-05 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142890+0000", 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3047000000000001e-05 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.143004+0000", 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113439 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.143177+0000", 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173365 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.143393+0000", 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216195 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.144561+0000", 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011678249999999999 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146103+0000", 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015422389999999999 2024-09-16T11:13:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146187+0000", 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3522000000000004e-05 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146251+0000", 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4185999999999998e-05 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:13 4.11 e33/31 ondisk, result = 0)", 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.145977+0000", 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.293192755, 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028279800000000001, 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145977+0000", 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145977+0000", 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145978+0000", 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.437e-06 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145987+0000", 2024-09-16T11:13:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3359999999999995e-06 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146005+0000", 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8354000000000001e-05 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146014+0000", 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5499999999999995e-06 2024-09-16T11:13:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146077+0000", 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3801999999999997e-05 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146091+0000", 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.381e-05 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146260+0000", 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016850900000000001 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:16.456 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:16.457 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:16.518 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:16.513+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:16.549 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:16.567 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:16.568 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:16.568 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:16.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:16.589 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:16.605 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:16.619 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:16.613+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:16.625 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:16.653 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:16.675 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:16.682 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:16.682 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:16.692 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:16.696 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:16.699 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:16.699 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:16.699 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:16.700 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:16.700 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:16.700 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:16.700 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:16.700 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:16.721 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:16.716+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:16.788 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:16.788 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:16.797 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:16.821 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:16.816+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:16.899 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:16.899 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:16.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:16.921 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:16.917+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:17.023 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:17.017+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:17.054 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.054 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.063 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:17.123 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:17.117+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:17.164 INFO:teuthology.orchestra.run.smithi162.stdout:skipping zero length punch hole 2024-09-16T11:13:17.168 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.168 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:17.190 INFO:teuthology.orchestra.run.smithi162.stdout:3 write 0x6f2d92c thru 0x6f367ff (0x8ed4 bytes) 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:6 read 0x19d3002 thru 0x19d456b (0x156a bytes) 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:7 read 0x305df39 thru 0x306ce71 (0xef39 bytes) 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:9 read 0x3044e8a thru 0x304e99f (0x9b16 bytes) 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:11 punch from 0x574ce1d to 0x5757f63, (0xb146 bytes) 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:12 punch from 0x14e6089 to 0x14ec093, (0x600a bytes) 2024-09-16T11:13:17.201 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0x3fa007c 2024-09-16T11:13:17.206 INFO:teuthology.orchestra.run.smithi162.stdout:13 trunc from 0x6f36800 to 0x3fa007c 2024-09-16T11:13:17.209 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:17.209 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:17.209 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:17.210 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-16T11:13:17.224 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:17.220+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:17.245 INFO:teuthology.orchestra.run.smithi162.stdout:14 writesame 0x6870aaf thru 0x6875295 data_size 0x1(0x47e7 bytes) 2024-09-16T11:13:17.245 INFO:teuthology.orchestra.run.smithi162.stdout:16 read 0x3e16ad5 thru 0x3e20e78 (0xa3a4 bytes) 2024-09-16T11:13:17.245 INFO:teuthology.orchestra.run.smithi162.stdout:17 write 0x5fc99ff thru 0x5fd5dfd (0xc3ff bytes) 2024-09-16T11:13:17.251 INFO:teuthology.orchestra.run.smithi162.stdout:18 writesame 0x13cc684 thru 0x13db408 data_size 0x1(0xed85 bytes) 2024-09-16T11:13:17.275 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.275 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.284 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:17.321 INFO:teuthology.orchestra.run.smithi162.stdout:19 writesame 0xd4a587b thru 0xd4a7256 data_size 0xa(0x19dc bytes) 2024-09-16T11:13:17.321 INFO:teuthology.orchestra.run.smithi162.stdout:20 punch from 0x7e1aacd to 0x7e23fbc, (0x94ef bytes) 2024-09-16T11:13:17.321 INFO:teuthology.orchestra.run.smithi162.stdout:21 write 0x605541c thru 0x605cf48 (0x7b2d bytes) 2024-09-16T11:13:17.325 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:17.321+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:17.327 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.327 INFO:teuthology.orchestra.run.smithi162.stdout:24 read 0xa48cbd9 thru 0xa493564 (0x698c bytes) 2024-09-16T11:13:17.328 INFO:teuthology.orchestra.run.smithi162.stdout:25 writesame 0xc2d4785 thru 0xc2e27f5 data_size 0x1(0xe071 bytes) 2024-09-16T11:13:17.373 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.373 INFO:teuthology.orchestra.run.smithi162.stdout:29 write 0x9ebacd6 thru 0x9ec6f72 (0xc29d bytes) 2024-09-16T11:13:17.380 INFO:teuthology.orchestra.run.smithi162.stdout:34 punch from 0x1e76c43 to 0x1e7abc7, (0x3f84 bytes) 2024-09-16T11:13:17.380 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0xc1117cb 2024-09-16T11:13:17.382 INFO:teuthology.orchestra.run.smithi162.stdout:35 trunc from 0xd4a7257 to 0xc1117cb 2024-09-16T11:13:17.398 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.413 INFO:teuthology.orchestra.run.smithi162.stdout:39 trunc from 0xc1117cb to 0x4650a48 2024-09-16T11:13:17.426 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:17.421+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:40 punch from 0x269070d to 0x26985ec, (0x7edf bytes) 2024-09-16T11:13:17.439 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.441 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.441 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.451 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:17.452 INFO:teuthology.orchestra.run.smithi162.stdout:42 write 0x5768600 thru 0x5775d64 (0xd765 bytes) 2024-09-16T11:13:17.471 INFO:teuthology.orchestra.run.smithi162.stdout:43 write 0x6f5ca1e thru 0x6f68a6a (0xc04d bytes) 2024-09-16T11:13:17.477 INFO:teuthology.orchestra.run.smithi162.stdout:44 trunc from 0x6f68a6b to 0x69c3b34 2024-09-16T11:13:17.497 INFO:teuthology.orchestra.run.smithi162.stdout:45 write 0xc091e6 thru 0xc13503 (0xa31e bytes) 2024-09-16T11:13:17.513 INFO:teuthology.orchestra.run.smithi162.stdout:46 trunc from 0x69c3b34 to 0xbb5c48e 2024-09-16T11:13:17.514 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:13:17.509+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e35 definitely_dead 0 2024-09-16T11:13:17.527 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:17.521+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:17.538 INFO:teuthology.orchestra.run.smithi162.stdout:48 write 0xcbf74b9 thru 0xcbfada0 (0x38e8 bytes) 2024-09-16T11:13:17.551 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.552 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.561 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:17.563 INFO:teuthology.orchestra.run.smithi162.stdout:51 trunc from 0xcbfada1 to 0x2882be1 2024-09-16T11:13:17.599 INFO:teuthology.orchestra.run.smithi162.stdout:52 write 0x76e4f20 thru 0x76e5e14 (0xef5 bytes) 2024-09-16T11:13:17.605 INFO:teuthology.orchestra.run.smithi162.stdout:53 write 0xb2327f thru 0xb2ef98 (0xbd1a bytes) 2024-09-16T11:13:17.611 INFO:teuthology.orchestra.run.smithi162.stdout:55 punch from 0x1650bbe to 0x16518b4, (0xcf6 bytes) 2024-09-16T11:13:17.611 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.628 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:17.624+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:17.633 INFO:teuthology.orchestra.run.smithi162.stdout:57 write 0xc28e976 thru 0xc295177 (0x6802 bytes) 2024-09-16T11:13:17.639 INFO:teuthology.orchestra.run.smithi162.stdout:58 write 0x4841bb7 thru 0x484e168 (0xc5b2 bytes) 2024-09-16T11:13:17.645 INFO:teuthology.orchestra.run.smithi162.stdout:59 write 0x9e26bae thru 0x9e3034c (0x979f bytes) 2024-09-16T11:13:17.650 INFO:teuthology.orchestra.run.smithi162.stdout:60 read 0x6df60a7 thru 0x6dfd7a5 (0x76ff bytes) 2024-09-16T11:13:17.650 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.662 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.662 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.665 INFO:teuthology.orchestra.run.smithi162.stdout:65 trunc from 0xc295178 to 0x584b1be 2024-09-16T11:13:17.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:17.711 INFO:teuthology.orchestra.run.smithi162.stdout:66 write 0xbe19dc5 thru 0xbe26041 (0xc27d bytes) 2024-09-16T11:13:17.717 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.724 INFO:teuthology.orchestra.run.smithi162.stdout:69 trunc from 0xbe26042 to 0x82f7d9c 2024-09-16T11:13:17.728 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:17.721+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:17.763 INFO:teuthology.orchestra.run.smithi162.stdout:70 write 0xbb0d905 thru 0xbb1bb73 (0xe26f bytes) 2024-09-16T11:13:17.769 INFO:teuthology.orchestra.run.smithi162.stdout:72 writesame 0xc54fc1 thru 0xc568e2 data_size 0x1(0x1922 bytes) 2024-09-16T11:13:17.779 INFO:teuthology.orchestra.run.smithi162.stdout:73 write 0x98fccc1 thru 0x9906a6e (0x9dae bytes) 2024-09-16T11:13:17.785 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.785 INFO:teuthology.orchestra.run.smithi162.stdout:75 writesame 0x22c18b5 thru 0x22c6e48 data_size 0x2(0x5594 bytes) 2024-09-16T11:13:17.802 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.802 INFO:teuthology.orchestra.run.smithi162.stdout:77 read 0x1691584 thru 0x169505e (0x3adb bytes) 2024-09-16T11:13:17.809 INFO:teuthology.orchestra.run.smithi162.stdout:78 trunc from 0xbb1bb74 to 0x87ec552 2024-09-16T11:13:17.823 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.823 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.829 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:17.825+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:17.830 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:17.830 INFO:teuthology.orchestra.run.smithi162.stdout:82 write 0x58dbede thru 0x58e3aec (0x7c0f bytes) 2024-09-16T11:13:17.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:17.839 INFO:teuthology.orchestra.run.smithi162.stdout:83 punch from 0x67e0259 to 0x67e3d8d, (0x3b34 bytes) 2024-09-16T11:13:17.839 INFO:teuthology.orchestra.run.smithi162.stdout:84 read 0x4d11874 thru 0x4d1cf72 (0xb6ff bytes) 2024-09-16T11:13:17.839 INFO:teuthology.orchestra.run.smithi162.stdout:86 writesame 0x44fe9e3 thru 0x4500938 data_size 0x15(0x1f56 bytes) 2024-09-16T11:13:17.850 INFO:teuthology.orchestra.run.smithi162.stdout:89 clone 1 order 24 su 524288 sc 12 2024-09-16T11:13:17.928 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:17.929 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:17.930 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:17.924+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:17.937 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:18.031 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:18.025+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:18.045 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.045 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:18.132 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:18.125+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:18.142 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:13:18.144 INFO:teuthology.orchestra.run.smithi162.stderr:marked out osd.2. 2024-09-16T11:13:18.164 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.164 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:18.233 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:18.228+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:18.318 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.318 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.327 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:18.334 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:18.328+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:18.422 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.422 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.432 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:18.434 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:18.428+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:18.528 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.528 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.536 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:18.532+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:18.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:18.636 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:18.632+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:18.640 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.641 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:18.710 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:18.738 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:18.732+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:18.762 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.763 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.772 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:18.839 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:18.833+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:18.842 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:18.865 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:18.868 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:18.868 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:18.877 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:18.881 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:18.912 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:18.940 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:18.936+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:18.941 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:18.962 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:18.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.142820046000001, 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:18.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:18.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:18.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.8522452760000001, 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:18.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:18.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:18.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:18.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:18.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.8438369880000001, 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:18.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.990 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:18.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:18.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:18.991 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:18.992 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:18.992 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:19.007 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:19.010 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:19.040 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:19.037+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:19.089 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:19.090 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:19.090 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:19.098 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:19.141 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:19.137+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:19.156 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:13:19.211 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:19.211 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:19.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:19.242 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:19.237+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:19.318 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:19.318 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:9 4.1a e34/31 4:5cea7035:::rbd_info:head v 34'2, mlcod=0'0)", 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.115686+0000", 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.2014869480000003, 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022638939999999998, 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115686+0000", 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115686+0000", 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115687+0000", 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115695+0000", 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6639999999999998e-06 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115720+0000", 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5182999999999999e-05 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115729+0000", 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5980000000000003e-06 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115852+0000", 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000122951 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:19.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.115893+0000", 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0744999999999997e-05 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.117898+0000", 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020051689999999998 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.117950+0000", 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2120000000000002e-05 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:19.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13 4.11 4:8e779da3:::rbd_object_map.11b5175acf49:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.142847+0000", 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.1743265909999998, 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003404327, 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13 2024-09-16T11:13:19.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142847+0000", 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142847+0000", 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142849+0000", 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.317e-06 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142859+0000", 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5389999999999996e-06 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142877+0000", 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8652999999999999e-05 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.142890+0000", 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3047000000000001e-05 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.143004+0000", 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113439 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.143177+0000", 2024-09-16T11:13:19.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000173365 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.143393+0000", 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216195 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.144561+0000", 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011678249999999999 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146103+0000", 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015422389999999999 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146187+0000", 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3522000000000004e-05 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146251+0000", 2024-09-16T11:13:19.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4185999999999998e-05 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:13 4.11 e33/31 ondisk, result = 0)", 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.145977+0000", 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.1711965949999996, 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028279800000000001, 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:19.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145977+0000", 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145977+0000", 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:19.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145978+0000", 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.437e-06 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.145987+0000", 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3359999999999995e-06 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146005+0000", 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8354000000000001e-05 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146014+0000", 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5499999999999995e-06 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146077+0000", 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3801999999999997e-05 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146091+0000", 2024-09-16T11:13:19.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.381e-05 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.146260+0000", 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016850900000000001 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:19.334 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:19.335 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:19.380 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:19.377+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:19.388 INFO:teuthology.orchestra.run.smithi162.stdout:90 writesame 0x48cbe3e thru 0x48d813b data_size 0xb(0xc2fe bytes) 2024-09-16T11:13:19.410 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:19.410 INFO:teuthology.orchestra.run.smithi162.stdout:93 punch from 0x36cefc1 to 0x36dd96f, (0xe9ae bytes) 2024-09-16T11:13:19.410 INFO:teuthology.orchestra.run.smithi162.stdout:94 read 0x6785cba thru 0x67959e3 (0xfd2a bytes) 2024-09-16T11:13:19.410 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:19.410 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:19.411 INFO:teuthology.orchestra.run.smithi162.stdout:100 read 0x60e624d thru 0x60ee97f (0x8733 bytes) 2024-09-16T11:13:19.414 INFO:teuthology.orchestra.run.smithi162.stdout:101 trunc from 0x87ec552 to 0x9be84bc 2024-09-16T11:13:19.445 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:19.441+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:19.447 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:19.447 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:19.452 INFO:teuthology.orchestra.run.smithi162.stdout:103 write 0xd578a21 thru 0xd57bd7c (0x335c bytes) 2024-09-16T11:13:19.457 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:19.470 INFO:teuthology.orchestra.run.smithi162.stdout:104 writesame 0xd6f7454 thru 0xd700602 data_size 0x1(0x91af bytes) 2024-09-16T11:13:19.470 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:19.470 INFO:teuthology.orchestra.run.smithi162.stdout:106 clone 2 order 25 su 8388608 sc 10 2024-09-16T11:13:19.546 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:19.541+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:19.549 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:19.549 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:19.549 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:19.549 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:19.549 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:19.557 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:19.647 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:19.641+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:9 4.1a 4:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.5473756099999996, 2024-09-16T11:13:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080919660000000008, 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 9 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108611+0000", 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4970000000000001e-06 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108629+0000", 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8527999999999999e-05 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108666+0000", 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6965000000000003e-05 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108686+0000", 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9457999999999999e-05 2024-09-16T11:13:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108888+0000", 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020222999999999999 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108902+0000", 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4511999999999999e-05 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.112496+0000", 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035939050000000001 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.112726+0000", 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000229559 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.113104+0000", 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000377592 2024-09-16T11:13:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.114998+0000", 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018945019999999999 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116382+0000", 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001384159 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116556+0000", 2024-09-16T11:13:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000174289 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116699+0000", 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014276999999999999 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:12 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.137212+0000", 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.5187705229999997, 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026444440000000001, 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 12 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137212+0000", 2024-09-16T11:13:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137212+0000", 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137215+0000", 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4210000000000002e-06 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137225+0000", 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0531e-05 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137250+0000", 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5103999999999999e-05 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137260+0000", 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0078000000000001e-05 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137351+0000", 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0801e-05 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137608+0000", 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025622999999999998 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.137966+0000", 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035821200000000001 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.139372+0000", 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014065950000000001 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.139720+0000", 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034799899999999999 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.139793+0000", 2024-09-16T11:13:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2452000000000004e-05 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.139857+0000", 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4021000000000002e-05 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:20 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.153532+0000", 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.5024504509999996, 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00071919699999999998, 2024-09-16T11:13:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 20 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153532+0000", 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153532+0000", 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153534+0000", 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.186e-06 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153539+0000", 2024-09-16T11:13:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6130000000000003e-06 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153556+0000", 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7173999999999999e-05 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153563+0000", 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6030000000000001e-06 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153632+0000", 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9470999999999999e-05 2024-09-16T11:13:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.153732+0000", 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9851000000000003e-05 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.154252+0000", 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051929899999999998 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:25 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [watch watch cookie 140364698725376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.157387+0000", 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.4985954069999998, 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001950186, 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 25 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157387+0000", 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157387+0000", 2024-09-16T11:13:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157389+0000", 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6869999999999999e-06 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157411+0000", 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1767e-05 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157442+0000", 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0697000000000001e-05 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157460+0000", 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8598e-05 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157545+0000", 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5062000000000007e-05 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157620+0000", 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4684000000000005e-05 2024-09-16T11:13:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.157777+0000", 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015692399999999999 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.158593+0000", 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00081574100000000004 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.159128+0000", 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053569000000000002 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.159196+0000", 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7983000000000003e-05 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.159338+0000", 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014135299999999999 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:314 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.170863+0000", 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.48511955699999998, 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00079681700000000001, 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 314 2024-09-16T11:13:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170863+0000", 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170863+0000", 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170865+0000", 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0930000000000001e-06 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170874+0000", 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8729999999999996e-06 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170892+0000", 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7713000000000001e-05 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170900+0000", 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1100000000000003e-06 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.170988+0000", 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7703999999999999e-05 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.171114+0000", 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012584200000000001 2024-09-16T11:13:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.171660+0000", 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054648200000000004 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:320 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.176019+0000", 2024-09-16T11:13:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.47996432700000002, 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024039769999999999, 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 320 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176019+0000", 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176019+0000", 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176020+0000", 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3820000000000001e-06 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176028+0000", 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7219999999999999e-06 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176046+0000", 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8683000000000001e-05 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176057+0000", 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0635000000000001e-05 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176129+0000", 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2317999999999994e-05 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176226+0000", 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6749000000000005e-05 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.176487+0000", 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026091100000000001 2024-09-16T11:13:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177305+0000", 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00081784900000000005 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.178282+0000", 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00097756199999999996 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.178360+0000", 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7484000000000006e-05 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.178423+0000", 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2681999999999994e-05 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:331 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.191923+0000", 2024-09-16T11:13:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46406015699999997, 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073560200000000002, 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 331 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.191923+0000", 2024-09-16T11:13:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.191923+0000", 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.191924+0000", 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.421e-06 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.191931+0000", 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.238e-06 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.191952+0000", 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0911999999999999e-05 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.191960+0000", 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3379999999999999e-06 2024-09-16T11:13:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.192089+0000", 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012965900000000001 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.192217+0000", 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012744300000000001 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.192658+0000", 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044159100000000002 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:360 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.241523+0000", 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41445948900000001, 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00096730000000000004, 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 360 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241523+0000", 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241523+0000", 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241526+0000", 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2340000000000001e-06 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241537+0000", 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1603000000000001e-05 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241569+0000", 2024-09-16T11:13:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2125000000000002e-05 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241597+0000", 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7410999999999999e-05 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241700+0000", 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010334400000000001 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.241869+0000", 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016900699999999999 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242491+0000", 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00062157599999999999 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:361 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.242813+0000", 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41316946900000001, 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00091671400000000001, 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 361 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242813+0000", 2024-09-16T11:13:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242813+0000", 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242816+0000", 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242841+0000", 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5213999999999999e-05 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242875+0000", 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4314999999999999e-05 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.242889+0000", 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3817e-05 2024-09-16T11:13:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.243052+0000", 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016296 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.243207+0000", 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015488700000000001 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.243730+0000", 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052331100000000005 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:365 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.246945+0000", 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.40903781900000002, 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000901737, 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 365 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.246945+0000", 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.246945+0000", 2024-09-16T11:13:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.246947+0000", 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2000000000000001e-06 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.246958+0000", 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0686999999999999e-05 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.246991+0000", 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3235000000000003e-05 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.247004+0000", 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.242e-05 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.247167+0000", 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000163074 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.247331+0000", 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016422600000000001 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.247847+0000", 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00051589500000000005 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:368 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.253428+0000", 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.40255527499999999, 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0010121349999999999, 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 368 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253428+0000", 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253428+0000", 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253430+0000", 2024-09-16T11:13:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3130000000000001e-06 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253442+0000", 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1936999999999999e-05 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253476+0000", 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4421999999999997e-05 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253506+0000", 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9417e-05 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253614+0000", 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108699 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.253787+0000", 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000173003 2024-09-16T11:13:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254440+0000", 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00065234400000000001 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:369 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.254793+0000", 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.40119006899999998, 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00092827800000000005, 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 369 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254793+0000", 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254793+0000", 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254795+0000", 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.193e-06 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254806+0000", 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0701e-05 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254839+0000", 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3358000000000001e-05 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.254856+0000", 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6793999999999999e-05 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.255026+0000", 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016979699999999999 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.255183+0000", 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000157606 2024-09-16T11:13:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.255721+0000", 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053782899999999998 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:373 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.259233+0000", 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.396750147, 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098177399999999993, 2024-09-16T11:13:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 373 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259233+0000", 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259233+0000", 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259235+0000", 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3070000000000001e-06 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259247+0000", 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1717e-05 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259280+0000", 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3185000000000002e-05 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259293+0000", 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.341e-05 2024-09-16T11:13:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259469+0000", 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017543699999999999 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.259654+0000", 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018515200000000001 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.260214+0000", 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00056056599999999997 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:375 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.352390+0000", 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.30359275699999999, 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020765570000000001, 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 375 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352390+0000", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352390+0000", 2024-09-16T11:13:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352393+0000", 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7800000000000001e-06 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352406+0000", 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2658e-05 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352441+0000", 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5744000000000002e-05 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352455+0000", 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3258000000000001e-05 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352557+0000", 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102367 2024-09-16T11:13:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.352738+0000", 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000181051 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.353048+0000", 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000310413 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354239+0000", 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011907160000000001 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354385+0000", 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014563 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354428+0000", 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3466e-05 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354467+0000", 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8473999999999998e-05 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:377 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [watch unwatch cookie 140364698725376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.354913+0000", 2024-09-16T11:13:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.30107028299999999, 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0026812210000000001, 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 377 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354913+0000", 2024-09-16T11:13:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354913+0000", 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354915+0000", 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0099999999999998e-06 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354934+0000", 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8944999999999999e-05 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354954+0000", 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0149999999999999e-05 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.354964+0000", 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0511e-05 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355104+0000", 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139302 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355242+0000", 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013893600000000001 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355466+0000", 2024-09-16T11:13:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022374299999999999 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.356598+0000", 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011319360000000001 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.357325+0000", 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00072641299999999997 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.357408+0000", 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3393999999999995e-05 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.357594+0000", 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018588099999999999 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:378 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.355270+0000", 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.300712545, 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024847760000000002, 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 378 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355270+0000", 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355270+0000", 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355273+0000", 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.174e-06 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355284+0000", 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0873000000000001e-05 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355316+0000", 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1262e-05 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355329+0000", 2024-09-16T11:13:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.377e-05 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.355943+0000", 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00061355100000000005 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for rw locks", 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.356007+0000", 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4208999999999995e-05 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.357653+0000", 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001646123 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.357704+0000", 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0367000000000001e-05 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.357755+0000", 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1446999999999997e-05 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:401 4.18 4:1ecbb4c5:::rbd_header.11b5175acf49:head [call rbd.snapshot_get in=15b,call rbd.parent_overlap_get in=21b,call rbd.get_flags in=12b,call rbd.get_protection_status in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.381181+0000", 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27480213599999997, 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090153800000000001, 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 401 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381181+0000", 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381181+0000", 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381182+0000", 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2079999999999999e-06 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381197+0000", 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5046e-05 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381216+0000", 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9125000000000001e-05 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381224+0000", 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381339+0000", 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115844 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.381526+0000", 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018621800000000001 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.382082+0000", 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00055666299999999999 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:415 4.13 4:cae7a2e7:::rbd_data.11b52605a025.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.400685+0000", 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.25529745399999998, 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0068934199999999999, 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 415 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400685+0000", 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400685+0000", 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400687+0000", 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3459999999999999e-06 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400694+0000", 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2919999999999998e-06 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400712+0000", 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7974000000000001e-05 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400719+0000", 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0339999999999999e-06 2024-09-16T11:13:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.400813+0000", 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3887000000000004e-05 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.401030+0000", 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021686899999999999 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.402192+0000", 2024-09-16T11:13:19.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011623709999999999 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407189+0000", 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049971479999999999 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407310+0000", 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000120637 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407449+0000", 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013924 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407579+0000", 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000129622 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:415 4.13 e37/31 ondisk, result = 0)", 2024-09-16T11:13:19.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.405637+0000", 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.25034594300000002, 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0015621019999999999, 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.405637+0000", 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.405637+0000", 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.405638+0000", 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08e-06 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.405736+0000", 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7795999999999997e-05 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.405754+0000", 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8527000000000001e-05 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.405770+0000", 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5342e-05 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407157+0000", 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00138759 2024-09-16T11:13:19.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407170+0000", 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2660000000000001e-05 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.407199+0000", 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9107000000000001e-05 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:442 4.d 4:b264cd0d:::rbd_data.11b52605a025.0000000000000012:head [set-alloc-hint object_size 16777216 write_size 16777216,write 2066977~13148] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.453735+0000", 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.20224811300000001, 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034035269999999999, 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 442 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453735+0000", 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453735+0000", 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453737+0000", 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4269999999999998e-06 2024-09-16T11:13:19.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453767+0000", 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9428e-05 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453801+0000", 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4424000000000001e-05 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453814+0000", 2024-09-16T11:13:19.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3234999999999999e-05 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.453939+0000", 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012476100000000001 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.454210+0000", 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027133799999999998 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:19.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.454461+0000", 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025036999999999998 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.455777+0000", 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001316439 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.456854+0000", 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001076933 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:19.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.457015+0000", 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016133400000000001 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.457138+0000", 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012283800000000001 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:19.768 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:19.770 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:19.770 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:19.765+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:19.871 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:19.865+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:19.917 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:19.918 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:19.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:19.972 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:19.965+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:20.026 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.027 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:20.027 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:20.027 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:20.027 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:20.036 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:20.084 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:20.081+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.305964745000001, 2024-09-16T11:13:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091644129999999997, 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822531+0000", 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1739999999999999e-06 2024-09-16T11:13:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822539+0000", 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4050000000000003e-06 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822554+0000", 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5384e-05 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822564+0000", 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7510000000000003e-06 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822695+0000", 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131481 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822720+0000", 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5069999999999999e-05 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826480+0000", 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037597590000000001 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826680+0000", 2024-09-16T11:13:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000200304 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 2", 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826846+0000", 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016532299999999999 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828757+0000", 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001911598 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831554+0000", 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027972930000000002 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831613+0000", 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8715000000000003e-05 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831690+0000", 2024-09-16T11:13:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7156000000000005e-05 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4276.0:1 2.3 e19/18 ondisk, result = 0)", 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:54.828577+0000", 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.299913786000001, 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018652799999999999, 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828577+0000", 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828577+0000", 2024-09-16T11:13:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828578+0000", 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1119999999999999e-06 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828594+0000", 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5993000000000001e-05 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828614+0000", 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9562e-05 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828653+0000", 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9638999999999998e-05 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828734+0000", 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0531999999999994e-05 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828746+0000", 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2055999999999999e-05 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828763+0000", 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7634000000000001e-05 2024-09-16T11:13:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:54.834712+0000", 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.293778705000001, 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022035800000000001, 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834712+0000", 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834712+0000", 2024-09-16T11:13:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834713+0000", 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.443e-06 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834731+0000", 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7591000000000001e-05 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834749+0000", 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8481000000000001e-05 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834759+0000", 2024-09-16T11:13:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7200000000000001e-06 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834817+0000", 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7661999999999998e-05 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.834932+0000", 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115461 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:54.835153+0000", 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.293337432000001, 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028001099999999999, 2024-09-16T11:13:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 3 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835153+0000", 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835153+0000", 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835155+0000", 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3850000000000001e-06 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835159+0000", 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4950000000000002e-06 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835173+0000", 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4337e-05 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835185+0000", 2024-09-16T11:13:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1606999999999999e-05 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835259+0000", 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3984000000000002e-05 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.835433+0000", 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000174203 2024-09-16T11:13:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:55.828079+0000", 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.300411732000001, 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061585909999999997, 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 5 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828079+0000", 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828079+0000", 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828082+0000", 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2770000000000001e-06 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828107+0000", 2024-09-16T11:13:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4646000000000001e-05 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828144+0000", 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7126000000000003e-05 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828163+0000", 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8660000000000001e-05 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828283+0000", 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012046 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.828294+0000", 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0664e-05 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.831472+0000", 2024-09-16T11:13:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003178476 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.831718+0000", 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024524900000000001 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.831948+0000", 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023080099999999999 2024-09-16T11:13:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833581+0000", 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001632664 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.834101+0000", 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052034100000000001 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.834179+0000", 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7780000000000004e-05 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.834238+0000", 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8446999999999998e-05 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4276.0:5 2.2 e19/18 ondisk, result = 0)", 2024-09-16T11:13:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:55.833905+0000", 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.294585186999999, 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034510299999999998, 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833905+0000", 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833905+0000", 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833907+0000", 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833916+0000", 2024-09-16T11:13:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8990000000000007e-06 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833943+0000", 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.741e-05 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.833964+0000", 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0581000000000002e-05 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.834073+0000", 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108971 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.834088+0000", 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4867e-05 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:55.834251+0000", 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016274400000000001 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:56.838129+0000", 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.29036211, 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028713810000000001, 2024-09-16T11:13:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 7 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838129+0000", 2024-09-16T11:13:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838129+0000", 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838132+0000", 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.977e-06 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838144+0000", 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2523e-05 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838177+0000", 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3422e-05 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838191+0000", 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3716999999999999e-05 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838343+0000", 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015215000000000001 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.838611+0000", 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026807099999999999 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.839018+0000", 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040685100000000001 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840887+0000", 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018683739999999999 2024-09-16T11:13:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840900+0000", 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2952000000000001e-05 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840952+0000", 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2330999999999998e-05 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.841000+0000", 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8013000000000003e-05 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4276.0:7 2.2 e20/18 ondisk, result = 0)", 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:56.840748+0000", 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.287742574999999, 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000141884, 2024-09-16T11:13:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840748+0000", 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840748+0000", 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840750+0000", 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.683e-06 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840760+0000", 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0139999999999999e-05 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840792+0000", 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1982e-05 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840803+0000", 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1450999999999999e-05 2024-09-16T11:13:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840870+0000", 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6649000000000005e-05 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840879+0000", 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9139999999999997e-06 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:56.840890+0000", 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1065e-05 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:11 4.1f e34/31 4:fa654f18:::rbd_id.image_client.0:head v 34'1, mlcod=0'0)", 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.134853+0000", 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9936379129999997, 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017562529999999999, 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134853+0000", 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134853+0000", 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134854+0000", 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2200000000000003e-07 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134863+0000", 2024-09-16T11:13:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6490000000000003e-06 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134896+0000", 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2684000000000001e-05 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134906+0000", 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.01e-05 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134977+0000", 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1359000000000006e-05 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.135027+0000", 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9929000000000003e-05 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136550+0000", 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001522357 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136609+0000", 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9352999999999999e-05 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.177459+0000", 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.95103191200000003, 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023200200000000001, 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177459+0000", 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177459+0000", 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177459+0000", 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177465+0000", 2024-09-16T11:13:20.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177471+0000", 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9560000000000002e-06 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177480+0000", 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5590000000000006e-06 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177552+0000", 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2559999999999996e-05 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.177691+0000", 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000138436 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:341 4.2 e37/31 4:4550e1e5:::rbd_id.image_client.0-clone1:head v 37'1, mlcod=0'0)", 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:19.201626+0000", 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.92686443100000004, 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001520488, 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201626+0000", 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201626+0000", 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201627+0000", 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.13e-07 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201635+0000", 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7640000000000005e-06 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201669+0000", 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4668000000000001e-05 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201683+0000", 2024-09-16T11:13:20.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3314e-05 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201802+0000", 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000119321 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.201846+0000", 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4261999999999997e-05 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.203086+0000", 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001239675 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:19.203147+0000", 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0871e-05 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.196 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.196 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:20.197 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:20.198 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:20.193+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:20.287 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone1 from 0xd700603 (overlap 0x87ec552) to 0x6cbd10e 2024-09-16T11:13:20.298 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:20.293+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:20.314 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:595 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0093564089999999996, 2024-09-16T11:13:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093775360000000006, 2024-09-16T11:13:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:13:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 595 2024-09-16T11:13:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2999999999999999e-07 2024-09-16T11:13:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301240+0000", 2024-09-16T11:13:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8359999999999999e-06 2024-09-16T11:13:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301269+0000", 2024-09-16T11:13:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8459e-05 2024-09-16T11:13:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301278+0000", 2024-09-16T11:13:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7949999999999996e-06 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304969+0000", 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003691864 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305136+0000", 2024-09-16T11:13:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016632999999999999 2024-09-16T11:13:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305576+0000", 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043990000000000001 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309262+0000", 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036865930000000002 2024-09-16T11:13:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:597 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0092144259999999995, 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092868369999999992, 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 597 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301378+0000", 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.657e-06 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301387+0000", 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3860000000000007e-06 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301402+0000", 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4875000000000001e-05 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301411+0000", 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5820000000000008e-06 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305997+0000", 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045854169999999996 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306078+0000", 2024-09-16T11:13:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1614999999999997e-05 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306441+0000", 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036263000000000002 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309329+0000", 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028879650000000002 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:598 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0091307860000000001, 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092455530000000001, 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 598 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301462+0000", 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301470+0000", 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6279999999999996e-06 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301484+0000", 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4314999999999999e-05 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301512+0000", 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8218999999999999e-05 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307142+0000", 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056298169999999996 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307264+0000", 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121704 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307701+0000", 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043725099999999999 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:599 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0089195200000000002, 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090760200000000006, 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 599 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301673+0000", 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3459999999999999e-06 2024-09-16T11:13:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301681+0000", 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0949999999999996e-06 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301696+0000", 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4932e-05 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301705+0000", 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6009999999999995e-06 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308183+0000", 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064783840000000002 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308260+0000", 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6601999999999996e-05 2024-09-16T11:13:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308708+0000", 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044877600000000001 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:601 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0084532240000000005, 2024-09-16T11:13:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086588979999999999, 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 601 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302139+0000", 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302152+0000", 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3608000000000001e-05 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302163+0000", 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1012000000000001e-05 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302169+0000", 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1309999999999997e-06 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309509+0000", 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073404869999999997 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309603+0000", 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4374000000000001e-05 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310124+0000", 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052012300000000005 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:603 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0077677839999999998, 2024-09-16T11:13:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080390779999999998, 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 603 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302824+0000", 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.06e-06 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302830+0000", 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9209999999999997e-06 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302844+0000", 2024-09-16T11:13:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3304e-05 2024-09-16T11:13:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302850+0000", 2024-09-16T11:13:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310529+0000", 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076792980000000002 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310601+0000", 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1070999999999998e-05 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:604 4.a 4.d77c2fea (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.006961788, 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072900650000000001, 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 604 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303631+0000", 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5579999999999999e-06 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303638+0000", 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303654+0000", 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6628000000000001e-05 2024-09-16T11:13:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303662+0000", 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8390000000000007e-06 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:605 4.13 e38/31 4:cae7a2e7:::rbd_data.11b52605a025.0000000000000001:head v 38'7, mlcod=35'4)", 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.0050299209999999997, 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0053951160000000001, 2024-09-16T11:13:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305562+0000", 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305567+0000", 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5949999999999998e-06 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305577+0000", 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305584+0000", 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3670000000000002e-06 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305659+0000", 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5593999999999995e-05 2024-09-16T11:13:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305672+0000", 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3193000000000001e-05 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:598 4.a e38/31)", 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.309622+0000", 2024-09-16T11:13:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.00096861099999999997, 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001378207, 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309622+0000", 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309622+0000", 2024-09-16T11:13:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309623+0000", 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8700000000000005e-07 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309628+0000", 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6029999999999998e-06 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309638+0000", 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9629999999999993e-06 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309643+0000", 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4469999999999997e-06 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:599 4.a e38/31)", 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.310087+0000", 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.00050420599999999999, 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095923099999999995, 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310087+0000", 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310087+0000", 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310087+0000", 2024-09-16T11:13:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310092+0000", 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7210000000000002e-06 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310106+0000", 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3713000000000001e-05 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310111+0000", 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3349999999999997e-06 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:13:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 10 2024-09-16T11:13:20.366 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:20.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:20.412 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:20.409+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:20.476 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.476 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:20.476 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:20.476 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:20.476 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:20.487 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:20.490 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T11:13:20.499 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:20.493+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:20.586 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.4498863819999999, 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066095169999999996, 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 11 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130364+0000", 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2200000000000001e-06 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130378+0000", 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3498e-05 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130415+0000", 2024-09-16T11:13:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6928999999999998e-05 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130431+0000", 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5834000000000001e-05 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130591+0000", 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016076400000000001 2024-09-16T11:13:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130610+0000", 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8244999999999999e-05 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.133935+0000", 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033251650000000002 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134174+0000", 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023965400000000001 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134553+0000", 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037902299999999998 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136098+0000", 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0015447010000000001 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136814+0000", 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00071608299999999995 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136888+0000", 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3526e-05 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136971+0000", 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2874999999999995e-05 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:456 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [read 0~40] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.158798+0000", 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.42144952499999999, 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037971150000000002, 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 456 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158798+0000", 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158798+0000", 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158801+0000", 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9019999999999999e-06 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158812+0000", 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1301000000000001e-05 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158853+0000", 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1032000000000003e-05 2024-09-16T11:13:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158869+0000", 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5556000000000001e-05 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.159009+0000", 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014004999999999999 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.159036+0000", 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7577999999999999e-05 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.162236+0000", 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031991789999999999 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.162312+0000", 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6402000000000005e-05 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.162595+0000", 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028311500000000002 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:537 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.285597+0000", 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.29465059399999999, 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035981170000000001, 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 537 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285597+0000", 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285597+0000", 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285600+0000", 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7499999999999999e-06 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285613+0000", 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.362e-05 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285653+0000", 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9688999999999999e-05 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285668+0000", 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5034e-05 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285797+0000", 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128582 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285968+0000", 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000171483 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.286616+0000", 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00064846299999999995 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.288322+0000", 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017059849999999999 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.288915+0000", 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059270999999999998 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.289084+0000", 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016872999999999999 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.289195+0000", 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011107099999999999 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:544 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.28287187499999999, 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073024969999999998, 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 544 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297378+0000", 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.99e-06 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297388+0000", 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0407e-05 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297415+0000", 2024-09-16T11:13:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7498e-05 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297426+0000", 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0957e-05 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297594+0000", 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167157 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297849+0000", 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025576 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.298833+0000", 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098353599999999992 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300723+0000", 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001890395 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304585+0000", 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038620709999999999 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304639+0000", 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3193999999999999e-05 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304678+0000", 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9532e-05 2024-09-16T11:13:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:549 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.28244350099999999, 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007009576, 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 549 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297806+0000", 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.747e-06 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297814+0000", 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2900000000000002e-06 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297858+0000", 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4295000000000002e-05 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297868+0000", 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2639999999999997e-06 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299859+0000", 2024-09-16T11:13:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019912609999999998 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299944+0000", 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5069999999999997e-05 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300312+0000", 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036818899999999999 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304727+0000", 2024-09-16T11:13:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044150689999999998 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304734+0000", 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304779+0000", 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5055000000000001e-05 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304814+0000", 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4770000000000001e-05 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:591 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27964971300000002, 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043393090000000004, 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 591 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300600+0000", 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1320000000000001e-06 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300611+0000", 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0961000000000001e-05 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300633+0000", 2024-09-16T11:13:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1999000000000001e-05 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300644+0000", 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1632e-05 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300770+0000", 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012577899999999999 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300927+0000", 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000157186 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301666+0000", 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073808200000000004 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304821+0000", 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031557349999999998 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304859+0000", 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7450999999999998e-05 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304902+0000", 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3579999999999999e-05 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304937+0000", 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4771999999999998e-05 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:592 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27936887300000002, 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061751929999999998, 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 592 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300880+0000", 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6080000000000001e-06 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300890+0000", 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4040000000000001e-06 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300907+0000", 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6903000000000001e-05 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300916+0000", 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5219999999999996e-06 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302514+0000", 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00159807 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302687+0000", 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000173155 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303339+0000", 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0006514 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306894+0000", 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035556720000000002 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306940+0000", 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5893999999999998e-05 2024-09-16T11:13:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307005+0000", 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4432999999999998e-05 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307054+0000", 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9132000000000003e-05 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:593 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27906371899999999, 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082575289999999996, 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 593 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301185+0000", 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2860000000000001e-06 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301193+0000", 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.678e-06 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301209+0000", 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5797000000000001e-05 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301218+0000", 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1980000000000007e-06 2024-09-16T11:13:20.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303958+0000", 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027405810000000002 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304015+0000", 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7074000000000003e-05 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304249+0000", 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023344199999999999 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307114+0000", 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028655569999999999 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309341+0000", 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022266339999999999 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309401+0000", 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0436000000000002e-05 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309441+0000", 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9845999999999998e-05 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:595 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.634 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27901280299999998, 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010351252, 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 595 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2999999999999999e-07 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301240+0000", 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8359999999999999e-06 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301269+0000", 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8459e-05 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301278+0000", 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7949999999999996e-06 2024-09-16T11:13:20.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304969+0000", 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003691864 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305136+0000", 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016632999999999999 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305576+0000", 2024-09-16T11:13:20.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043990000000000001 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309262+0000", 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036865930000000002 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311524+0000", 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022620909999999999 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311559+0000", 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4597000000000001e-05 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311586+0000", 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6957e-05 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:597 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27887082000000002, 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010265962, 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.641 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 597 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301378+0000", 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.657e-06 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301387+0000", 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3860000000000007e-06 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301402+0000", 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4875000000000001e-05 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301411+0000", 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5820000000000008e-06 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305997+0000", 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045854169999999996 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306078+0000", 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1614999999999997e-05 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306441+0000", 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036263000000000002 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309329+0000", 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028879650000000002 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311596+0000", 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022673939999999998 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311621+0000", 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4797e-05 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311643+0000", 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1644000000000001e-05 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:598 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:20.647 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27878718000000002, 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010230927000000001, 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 598 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301462+0000", 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301470+0000", 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6279999999999996e-06 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301484+0000", 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4314999999999999e-05 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301512+0000", 2024-09-16T11:13:20.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8218999999999999e-05 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307142+0000", 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056298169999999996 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307264+0000", 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121704 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307701+0000", 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043725099999999999 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311457+0000", 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037559 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311649+0000", 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019221900000000001 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311673+0000", 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3603000000000001e-05 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311691+0000", 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8609000000000001e-05 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:599 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27857591399999998, 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014841653999999999, 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 599 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:20.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:20.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301673+0000", 2024-09-16T11:13:20.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3459999999999999e-06 2024-09-16T11:13:20.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301681+0000", 2024-09-16T11:13:20.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0949999999999996e-06 2024-09-16T11:13:20.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301696+0000", 2024-09-16T11:13:20.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4932e-05 2024-09-16T11:13:20.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301705+0000", 2024-09-16T11:13:20.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6009999999999995e-06 2024-09-16T11:13:20.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308183+0000", 2024-09-16T11:13:20.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064783840000000002 2024-09-16T11:13:20.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308260+0000", 2024-09-16T11:13:20.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6601999999999996e-05 2024-09-16T11:13:20.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308708+0000", 2024-09-16T11:13:20.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044877600000000001 2024-09-16T11:13:20.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311516+0000", 2024-09-16T11:13:20.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028079189999999999 2024-09-16T11:13:20.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316392+0000", 2024-09-16T11:13:20.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004875735 2024-09-16T11:13:20.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316467+0000", 2024-09-16T11:13:20.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4571000000000001e-05 2024-09-16T11:13:20.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316513+0000", 2024-09-16T11:13:20.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6693000000000002e-05 2024-09-16T11:13:20.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.667 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.667 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.667 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:601 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.667 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:20.668 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.278109618, 2024-09-16T11:13:20.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014436584000000001, 2024-09-16T11:13:20.668 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.668 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.668 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.669 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 601 2024-09-16T11:13:20.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.669 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:20.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:20.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302139+0000", 2024-09-16T11:13:20.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:20.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302152+0000", 2024-09-16T11:13:20.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3608000000000001e-05 2024-09-16T11:13:20.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302163+0000", 2024-09-16T11:13:20.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1012000000000001e-05 2024-09-16T11:13:20.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302169+0000", 2024-09-16T11:13:20.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1309999999999997e-06 2024-09-16T11:13:20.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309509+0000", 2024-09-16T11:13:20.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073404869999999997 2024-09-16T11:13:20.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309603+0000", 2024-09-16T11:13:20.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4374000000000001e-05 2024-09-16T11:13:20.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310124+0000", 2024-09-16T11:13:20.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052012300000000005 2024-09-16T11:13:20.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312552+0000", 2024-09-16T11:13:20.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024287969999999999 2024-09-16T11:13:20.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316527+0000", 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039741569999999999 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316553+0000", 2024-09-16T11:13:20.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945999999999999e-05 2024-09-16T11:13:20.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316574+0000", 2024-09-16T11:13:20.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1937999999999999e-05 2024-09-16T11:13:20.680 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:603 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27742417800000002, 2024-09-16T11:13:20.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019179324000000001, 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 603 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302824+0000", 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.06e-06 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302830+0000", 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9209999999999997e-06 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302844+0000", 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3304e-05 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302850+0000", 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310529+0000", 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076792980000000002 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310601+0000", 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1070999999999998e-05 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311059+0000", 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045884099999999998 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313207+0000", 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002147805 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321862+0000", 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086545330000000007 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321919+0000", 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7130999999999999e-05 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322003+0000", 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3794000000000004e-05 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:604 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27661818199999999, 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.018458927999999999, 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 604 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303631+0000", 2024-09-16T11:13:20.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5579999999999999e-06 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303638+0000", 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303654+0000", 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6628000000000001e-05 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303662+0000", 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8390000000000007e-06 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311727+0000", 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080650300000000008 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311786+0000", 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8507000000000001e-05 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312144+0000", 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358821 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.314152+0000", 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002008045 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322013+0000", 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078606920000000007 2024-09-16T11:13:20.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322053+0000", 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322088+0000", 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4972000000000003e-05 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:605 4.13 e38/31 4:cae7a2e7:::rbd_data.11b52605a025.0000000000000001:head v 38'7, mlcod=35'4)", 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.27468631500000001, 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056889410000000003, 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305562+0000", 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305567+0000", 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5949999999999998e-06 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305577+0000", 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305584+0000", 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3670000000000002e-06 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305659+0000", 2024-09-16T11:13:20.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5593999999999995e-05 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305672+0000", 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3193000000000001e-05 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311201+0000", 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055283249999999997 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311250+0000", 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9382000000000003e-05 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:613 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'8, mlcod=37'5)", 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.26771879599999998, 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095140820000000001, 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312549+0000", 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9432999999999998e-05 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312559+0000", 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0455000000000001e-05 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312565+0000", 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4609999999999999e-06 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312594+0000", 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9456000000000001e-05 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312622+0000", 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7586999999999999e-05 2024-09-16T11:13:20.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322004+0000", 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093818349999999998 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322043+0000", 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9167e-05 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:614 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'10, mlcod=37'5)", 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.267227785, 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010261963000000001, 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313021+0000", 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5799999999999998e-07 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313028+0000", 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1400000000000002e-06 2024-09-16T11:13:20.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313041+0000", 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2958e-05 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313047+0000", 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0530000000000003e-06 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313090+0000", 2024-09-16T11:13:20.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2936999999999999e-05 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313105+0000", 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5199e-05 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323210+0000", 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010104958000000001 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323282+0000", 2024-09-16T11:13:20.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1860000000000007e-05 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:617 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.26346764900000003, 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060101130000000001, 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 617 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316781+0000", 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.279e-06 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316910+0000", 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012939700000000001 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316922+0000", 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1894e-05 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316928+0000", 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5559999999999998e-06 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316974+0000", 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5587000000000002e-05 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317094+0000", 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000120586 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317242+0000", 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147517 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322697+0000", 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054555960000000001 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322708+0000", 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0805000000000001e-05 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322745+0000", 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7280000000000002e-05 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322790+0000", 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4616000000000001e-05 2024-09-16T11:13:20.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:618 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.26330579300000001, 2024-09-16T11:13:20.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006757379, 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 618 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316943+0000", 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9199999999999999e-07 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:20.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316945+0000", 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5009999999999999e-06 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316952+0000", 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5980000000000002e-06 2024-09-16T11:13:20.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316955+0000", 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4879999999999999e-06 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318095+0000", 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011395050000000001 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318151+0000", 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6577999999999999e-05 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318284+0000", 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013229499999999999 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323278+0000", 2024-09-16T11:13:20.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049947200000000002 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323605+0000", 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032718800000000001 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323658+0000", 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2961999999999999e-05 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:20.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323699+0000", 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0652000000000003e-05 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:20.720 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:20.722 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:20.722 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:20.717+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:20.756 INFO:teuthology.orchestra.run.smithi162.stdout:107 write 0xb9bbd57 thru 0xb9c34eb (0x7795 bytes) 2024-09-16T11:13:20.775 INFO:teuthology.orchestra.run.smithi162.stdout:108 read 0x272c29a thru 0x272d384 (0x10eb bytes) 2024-09-16T11:13:20.781 INFO:teuthology.orchestra.run.smithi162.stdout:109 trunc from 0xd700603 to 0xbc7e697 2024-09-16T11:13:20.823 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:20.817+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:20.848 INFO:teuthology.orchestra.run.smithi162.stdout:110 read 0x14ee69b thru 0x14f327f (0x4be5 bytes) 2024-09-16T11:13:20.849 INFO:teuthology.orchestra.run.smithi162.stdout:112 punch from 0x89dd6e8 to 0x89e0fa1, (0x38b9 bytes) 2024-09-16T11:13:20.851 INFO:teuthology.orchestra.run.smithi162.stdout:113 write 0x64d1851 thru 0x64d3bca (0x237a bytes) 2024-09-16T11:13:20.875 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.875 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:20.876 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:20.876 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:20.887 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:20.918 INFO:teuthology.orchestra.run.smithi162.stdout:114 trunc from 0xbc7e697 to 0x348b8c3 2024-09-16T11:13:20.923 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:20.917+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:20.955 INFO:teuthology.orchestra.run.smithi162.stdout:116 punch from 0x4e2394 to 0x4eac94, (0x8900 bytes) 2024-09-16T11:13:20.961 INFO:teuthology.orchestra.run.smithi162.stdout:117 trunc from 0x348b8c3 to 0x62c4165 2024-09-16T11:13:20.972 INFO:teuthology.orchestra.run.smithi162.stdout:119 read 0x30948ba thru 0x3099a7a (0x51c1 bytes) 2024-09-16T11:13:20.972 INFO:teuthology.orchestra.run.smithi162.stdout:123 read 0x3d165c3 thru 0x3d1c3f3 (0x5e31 bytes) 2024-09-16T11:13:20.972 INFO:teuthology.orchestra.run.smithi162.stdout:124 read 0x3333122 thru 0x3335671 (0x2550 bytes) 2024-09-16T11:13:20.985 INFO:teuthology.orchestra.run.smithi162.stdout:125 trunc from 0x62c4165 to 0x2962e 2024-09-16T11:13:20.988 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:20.988 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:20.989 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:20.989 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:20.989 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:21.000 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:21.021 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:21.044 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:21.041+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:21.045 INFO:teuthology.orchestra.run.smithi162.stdout:126 write 0x990b23d thru 0x9917bc2 (0xc986 bytes) 2024-09-16T11:13:21.050 INFO:teuthology.orchestra.run.smithi162.stdout:129 read 0x7006f0b thru 0x700f81c (0x8912 bytes) 2024-09-16T11:13:21.050 INFO:teuthology.orchestra.run.smithi162.stdout:133 read 0x6006d51 thru 0x600a693 (0x3943 bytes) 2024-09-16T11:13:21.057 INFO:teuthology.orchestra.run.smithi162.stdout:134 trunc from 0x9917bc3 to 0x5ea5dd9 2024-09-16T11:13:21.070 INFO:teuthology.orchestra.run.smithi162.stdout:135 read 0x47a4d07 thru 0x47b1dd4 (0xd0ce bytes) 2024-09-16T11:13:21.070 INFO:teuthology.orchestra.run.smithi162.stdout:136 punch from 0x192130f to 0x192e209, (0xcefa bytes) 2024-09-16T11:13:21.070 INFO:teuthology.orchestra.run.smithi162.stdout:137 read 0x24cc296 thru 0x24dbaa0 (0xf80b bytes) 2024-09-16T11:13:21.070 INFO:teuthology.orchestra.run.smithi162.stdout:138 read 0x153df2e thru 0x1547cba (0x9d8d bytes) 2024-09-16T11:13:21.078 INFO:teuthology.orchestra.run.smithi162.stdout:141 trunc from 0x5ea5dd9 to 0x139c9b8 2024-09-16T11:13:21.125 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:21.121+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:21.132 INFO:teuthology.orchestra.run.smithi162.stdout:142 writesame 0xeb0765f thru 0xeb09a66 data_size 0x4(0x2408 bytes) 2024-09-16T11:13:21.132 INFO:teuthology.orchestra.run.smithi162.stdout:143 write 0xc820924 thru 0xc82814a (0x7827 bytes) 2024-09-16T11:13:21.137 INFO:teuthology.orchestra.run.smithi162.stdout:145 write 0x9557067 thru 0x9557f8b (0xf25 bytes) 2024-09-16T11:13:21.141 INFO:teuthology.orchestra.run.smithi162.stdout:146 writesame 0x2d3867c thru 0x2d3f3ab data_size 0x8(0x6d30 bytes) 2024-09-16T11:13:21.144 INFO:teuthology.orchestra.run.smithi162.stdout:147 punch from 0xe1eb95f to 0xe1f274e, (0x6def bytes) 2024-09-16T11:13:21.144 INFO:teuthology.orchestra.run.smithi162.stdout:148 punch from 0xd5691c8 to 0xd572450, (0x9288 bytes) 2024-09-16T11:13:21.145 INFO:teuthology.orchestra.run.smithi162.stdout:149 read 0x947c4e thru 0x9522b5 (0xa668 bytes) 2024-09-16T11:13:21.145 INFO:teuthology.orchestra.run.smithi162.stdout:151 read 0xcf4f2b5 thru 0xcf5db79 (0xe8c5 bytes) 2024-09-16T11:13:21.145 INFO:teuthology.orchestra.run.smithi162.stdout:153 read 0xa05c0b2 thru 0xa068aaf (0xc9fe bytes) 2024-09-16T11:13:21.145 INFO:teuthology.orchestra.run.smithi162.stdout:154 write 0xb6e4944 thru 0xb6eccd0 (0x838d bytes) 2024-09-16T11:13:21.149 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:21.150 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:21.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:21.150 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:21.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.151 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 31.367709278, 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.152 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:21.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:21.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:21.160 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979783577999999, 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:21.161 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:21.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:21.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:21.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:21.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:21.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.170 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979632248000001, 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.171 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:21.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:21.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.178 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979580762000001, 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:21.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:21.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:21.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979510613999999, 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:21.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.190 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979338144, 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.191 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:21.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979279821999999, 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:21.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:21.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.979127702, 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:21.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.208 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.978949183000001, 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:21.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.214 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.978916210000001, 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.215 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:21.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:21.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:21.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:21.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:21.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:21.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:21.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:21.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:21.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:21.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:21.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:21.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:21.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:21.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:21.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:21.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:21.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:21.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:21.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:21.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:21.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:21.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:21.226 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.226 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.227 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.227 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.227 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:21.227 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.426750718000001, 2024-09-16T11:13:21.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:21.228 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.228 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.228 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.228 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.228 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.229 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:21.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.229 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:21.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:21.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:21.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:21.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:21.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:21.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:21.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:21.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:21.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:21.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:21.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:21.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:21.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:21.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:21.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:21.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:21.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:21.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:21.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.240 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.241 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.241 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:21.241 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.426728485000002, 2024-09-16T11:13:21.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:21.242 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.242 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.242 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:21.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.243 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:21.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:21.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:21.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:21.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:21.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:21.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:21.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:21.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:21.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:21.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:21.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:21.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:21.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:21.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:21.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:21.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:21.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:21.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:21.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:21.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:21.254 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.426599688, 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:21.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:21.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:21.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.426541343, 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.263 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.264 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.264 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.264 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.264 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:21.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.264 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:21.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:21.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:21.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:21.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:21.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:21.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:21.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:21.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:21.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:21.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:21.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:21.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:21.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:21.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:21.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:21.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:21.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:21.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:21.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:21.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:21.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:21.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:21.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:21.276 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.276 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.276 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.277 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.277 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:21.277 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.426486954000001, 2024-09-16T11:13:21.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:21.277 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.278 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.278 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:21.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.279 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:21.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:21.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:21.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:21.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:21.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:21.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:21.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:21.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:21.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.286 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 30.426352647000002, 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.287 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:21.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:21.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.645255126999999, 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:21.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:21.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.298 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.645197278000001, 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:21.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:21.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.645109966, 2024-09-16T11:13:21.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:21.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.644871838, 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:21.310 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:21.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:21.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:21.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:21.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:21.316 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:21.316 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:13:21.318 INFO:teuthology.orchestra.run.smithi162.stdout:155 read 0x50fa59b thru 0x5105d85 (0xb7eb bytes) 2024-09-16T11:13:21.318 INFO:teuthology.orchestra.run.smithi162.stdout:156 writesame 0xd64876 thru 0xd66e01 data_size 0x36(0x258c bytes) 2024-09-16T11:13:21.318 INFO:teuthology.orchestra.run.smithi162.stdout:158 punch from 0x88f24f0 to 0x88f3c0a, (0x171a bytes) 2024-09-16T11:13:21.318 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:21.318 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:21.319 INFO:teuthology.orchestra.run.smithi162.stdout:159 trunc from 0xeb09a67 to 0x74f9a 2024-09-16T11:13:21.319 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:21.313+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:21.319 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:21.320 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:21.320 INFO:teuthology.orchestra.run.smithi162.stdout:161 write 0x62cb92 thru 0x636536 (0x99a5 bytes) 2024-09-16T11:13:21.320 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:21.320 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:21.321 INFO:teuthology.orchestra.run.smithi162.stdout:163 write 0xe23d499 thru 0xe2440da (0x6c42 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:164 write 0x41436fe thru 0x4152686 (0xef89 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:167 write 0xa6a6679 thru 0xa6a9899 (0x3221 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:168 read 0xa54263c thru 0xa548145 (0x5b0a bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:169 read 0x2d9bcd8 thru 0x2d9cf5f (0x1288 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:171 read 0x5d2a306 thru 0x5d39ee9 (0xfbe4 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:172 read 0x8367afb thru 0x8367f92 (0x498 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:175 read 0xca28e3 thru 0xcae973 (0xc091 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:177 write 0xd3dfaac thru 0xd3eed94 (0xf2e9 bytes) 2024-09-16T11:13:21.322 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:21.323 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:21.323 INFO:teuthology.orchestra.run.smithi162.stdout:181 read 0xb990ee5 thru 0xb993b57 (0x2c73 bytes) 2024-09-16T11:13:21.323 INFO:teuthology.orchestra.run.smithi162.stdout:182 read 0x5df7c88 thru 0x5e02cfe (0xb077 bytes) 2024-09-16T11:13:21.323 INFO:teuthology.orchestra.run.smithi162.stdout:183 read 0xca321a thru 0xcaf7ee (0xc5d5 bytes) 2024-09-16T11:13:21.323 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:21.324 INFO:teuthology.orchestra.run.smithi162.stdout:186 read 0x2478e69 thru 0x24817fa (0x8992 bytes) 2024-09-16T11:13:21.324 INFO:teuthology.orchestra.run.smithi162.stdout:187 write 0x3031912 thru 0x30399f1 (0x80e0 bytes) 2024-09-16T11:13:21.324 INFO:teuthology.orchestra.run.smithi162.stdout:188 write 0x58b4bc6 thru 0x58be30c (0x9747 bytes) 2024-09-16T11:13:21.324 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:21.324 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:21.324 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:21.325 INFO:teuthology.orchestra.run.smithi162.stdout:191 trunc from 0xe2440db to 0x24a2369 2024-09-16T11:13:21.328 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:21.330 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:21.331 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:21.345 INFO:teuthology.orchestra.run.smithi162.stdout:192 writesame 0x501a865 thru 0x5020544 data_size 0x10(0x5ce0 bytes) 2024-09-16T11:13:21.352 INFO:teuthology.orchestra.run.smithi162.stdout:193 trunc from 0x5020545 to 0x200eb19 2024-09-16T11:13:21.365 INFO:teuthology.orchestra.run.smithi162.stdout:194 write 0x83f8be thru 0x847a3e (0x8181 bytes) 2024-09-16T11:13:21.369 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:21.369 INFO:teuthology.orchestra.run.smithi162.stdout:196 clone 3 order 22 su 131072 sc 4 2024-09-16T11:13:21.419 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:21.413+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:21.422 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:21.422 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:21.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:21.520 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:21.513+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:21.573 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:21.573 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:21.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:21.621 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:21.617+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:21.681 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:21.681 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:21.690 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:21.723 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:21.717+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:21.785 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:21.785 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:21.794 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:21.823 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:21.817+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:21.890 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:21.890 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:21.900 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:21.924 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:21.921+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:22.001 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.001 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.010 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:22.025 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:22.021+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:22.109 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.109 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.117 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:22.126 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:22.121+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:22.172 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:13:22.223 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.224 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.228 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:22.221+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:22.233 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:22.296 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone2 from 0x200eb19 (overlap 0x2962e) to 0x211be 2024-09-16T11:13:22.330 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:22.325+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:22.337 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.338 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.347 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:22.426 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T11:13:22.431 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:22.425+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:22.457 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.458 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.468 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:22.531 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:22.525+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:22.610 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.610 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.620 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:22.632 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:22.629+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:22.646 INFO:teuthology.orchestra.run.smithi162.stdout:197 punch from 0x1de1ba5 to 0x1de72db, (0x5736 bytes) 2024-09-16T11:13:22.653 INFO:teuthology.orchestra.run.smithi162.stdout:198 punch from 0x1e020e2 to 0x1e0bc3f, (0x9b5d bytes) 2024-09-16T11:13:22.653 INFO:teuthology.orchestra.run.smithi162.stdout:199 read 0x198a4e0 thru 0x198ea70 (0x4591 bytes) 2024-09-16T11:13:22.654 INFO:teuthology.orchestra.run.smithi162.stdout:200 read 0xaf8110 thru 0xafd48f (0x5380 bytes) 2024-09-16T11:13:22.655 INFO:teuthology.orchestra.run.smithi162.stdout:202 trunc from 0x200eb19 to 0x1c6be41 2024-09-16T11:13:22.718 INFO:teuthology.orchestra.run.smithi162.stdout:203 write 0xb93b3b7 thru 0xb940476 (0x50c0 bytes) 2024-09-16T11:13:22.724 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.724 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.725 INFO:teuthology.orchestra.run.smithi162.stdout:205 trunc from 0xb940477 to 0xb9a2634 2024-09-16T11:13:22.733 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:22.729+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:22.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:22.736 INFO:teuthology.orchestra.run.smithi162.stdout:206 read 0x437ef7e thru 0x43850eb (0x616e bytes) 2024-09-16T11:13:22.736 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:22.736 INFO:teuthology.orchestra.run.smithi162.stdout:209 write 0x8c7b566 thru 0x8c83cc1 (0x875c bytes) 2024-09-16T11:13:22.742 INFO:teuthology.orchestra.run.smithi162.stdout:210 write 0x28164a6 thru 0x281b79f (0x52fa bytes) 2024-09-16T11:13:22.754 INFO:teuthology.orchestra.run.smithi162.stdout:211 trunc from 0xb9a2634 to 0x89f8bf7 2024-09-16T11:13:22.781 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:22.786 INFO:teuthology.orchestra.run.smithi162.stdout:213 trunc from 0x89f8bf7 to 0xabf9737 2024-09-16T11:13:22.799 INFO:teuthology.orchestra.run.smithi162.stdout:214 writesame 0x61ec71b thru 0x61f0d63 data_size 0x1(0x4649 bytes) 2024-09-16T11:13:22.834 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:22.829+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:22.837 INFO:teuthology.orchestra.run.smithi162.stdout:216 write 0xbec1d1d thru 0xbec4264 (0x2548 bytes) 2024-09-16T11:13:22.843 INFO:teuthology.orchestra.run.smithi162.stdout:217 writesame 0x750507b thru 0x750afe2 data_size 0x56(0x5f68 bytes) 2024-09-16T11:13:22.849 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.849 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.850 INFO:teuthology.orchestra.run.smithi162.stdout:220 writesame 0x9ed0a5e thru 0x9ed7c26 data_size 0x1(0x71c9 bytes) 2024-09-16T11:13:22.859 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:22.879 INFO:teuthology.orchestra.run.smithi162.stdout:221 punch from 0xae5317f to 0xae598f7, (0x6778 bytes) 2024-09-16T11:13:22.880 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0xd387fa8 2024-09-16T11:13:22.883 INFO:teuthology.orchestra.run.smithi162.stdout:222 trunc from 0xbec4265 to 0xd387fa8 2024-09-16T11:13:22.893 INFO:teuthology.orchestra.run.smithi162.stdout:223 read 0x5c0fcac thru 0x5c114aa (0x17ff bytes) 2024-09-16T11:13:22.893 INFO:teuthology.orchestra.run.smithi162.stdout:226 write 0x8791646 thru 0x87993c7 (0x7d82 bytes) 2024-09-16T11:13:22.895 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:22.896 INFO:teuthology.orchestra.run.smithi162.stdout:229 write 0xa6152fd thru 0xa616ab8 (0x17bc bytes) 2024-09-16T11:13:22.904 INFO:teuthology.orchestra.run.smithi162.stdout:230 read 0x5307fad thru 0x5316bf9 (0xec4d bytes) 2024-09-16T11:13:22.904 INFO:teuthology.orchestra.run.smithi162.stdout:231 writesame 0x91befeb thru 0x91cc0c4 data_size 0x86(0xd0da bytes) 2024-09-16T11:13:22.911 INFO:teuthology.orchestra.run.smithi162.stdout:232 writesame 0x61cdec6 thru 0x61d6ccb data_size 0x62(0x8e06 bytes) 2024-09-16T11:13:22.918 INFO:teuthology.orchestra.run.smithi162.stdout:233 write 0x8a95260 thru 0x8aa1570 (0xc311 bytes) 2024-09-16T11:13:22.924 INFO:teuthology.orchestra.run.smithi162.stdout:234 writesame 0x38da100 thru 0x38e6393 data_size 0x2a(0xc294 bytes) 2024-09-16T11:13:22.934 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:22.929+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:22.954 INFO:teuthology.orchestra.run.smithi162.stdout:235 trunc from 0xd387fa8 to 0x33e1aae 2024-09-16T11:13:22.965 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:22.965 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:22.974 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:22.977 INFO:teuthology.orchestra.run.smithi162.stdout:237 read 0xb7ac43 thru 0xb85c82 (0xb040 bytes) 2024-09-16T11:13:22.979 INFO:teuthology.orchestra.run.smithi162.stdout:238 read 0x6685c2 thru 0x66be95 (0x38d4 bytes) 2024-09-16T11:13:22.981 INFO:teuthology.orchestra.run.smithi162.stdout:239 trunc from 0x33e1aae to 0x27954a7 2024-09-16T11:13:23.035 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:23.033+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:23.045 INFO:teuthology.orchestra.run.smithi162.stdout:242 writesame 0xce80721 thru 0xce85784 data_size 0x14(0x5064 bytes) 2024-09-16T11:13:23.045 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:23.045 INFO:teuthology.orchestra.run.smithi162.stdout:247 write 0x5b554f8 thru 0x5b560d6 (0xbdf bytes) 2024-09-16T11:13:23.052 INFO:teuthology.orchestra.run.smithi162.stdout:250 writesame 0x163514f thru 0x163a1be data_size 0x4e(0x5070 bytes) 2024-09-16T11:13:23.054 INFO:teuthology.orchestra.run.smithi162.stdout:251 read 0x38da842 thru 0x38e70c2 (0xc881 bytes) 2024-09-16T11:13:23.055 INFO:teuthology.orchestra.run.smithi162.stdout:252 write 0x47141c9 thru 0x4716a5c (0x2894 bytes) 2024-09-16T11:13:23.060 INFO:teuthology.orchestra.run.smithi162.stdout:253 write 0x98198fa thru 0x981b634 (0x1d3b bytes) 2024-09-16T11:13:23.071 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:23.071 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:23.081 INFO:teuthology.orchestra.run.smithi162.stdout:254 write 0xeab8f39 thru 0xeabdadf (0x4ba7 bytes) 2024-09-16T11:13:23.082 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:23.087 INFO:teuthology.orchestra.run.smithi162.stdout:255 punch from 0x807eb4b to 0x8080cc3, (0x2178 bytes) 2024-09-16T11:13:23.087 INFO:teuthology.orchestra.run.smithi162.stdout:256 punch from 0x39886e4 to 0x39897b2, (0x10ce bytes) 2024-09-16T11:13:23.087 INFO:teuthology.orchestra.run.smithi162.stdout:257 write 0x2fc526c thru 0x2fcf2cc (0xa061 bytes) 2024-09-16T11:13:23.110 INFO:teuthology.orchestra.run.smithi162.stdout:258 trunc from 0xeabdae0 to 0x643bbab 2024-09-16T11:13:23.128 INFO:teuthology.orchestra.run.smithi162.stdout:259 writesame 0x5fde007 thru 0x5fe0be0 data_size 0x3(0x2bda bytes) 2024-09-16T11:13:23.136 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:23.133+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:23.142 INFO:teuthology.orchestra.run.smithi162.stdout:260 write 0x1aeb181 thru 0x1aebfc9 (0xe49 bytes) 2024-09-16T11:13:23.144 INFO:teuthology.orchestra.run.smithi162.stdout:261 write 0x214d6e5 thru 0x2155016 (0x7932 bytes) 2024-09-16T11:13:23.155 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:23.155 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:23.155 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:13:23.155 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-16T11:13:23.158 INFO:teuthology.orchestra.run.smithi162.stdout:262 trunc from 0x643bbab to 0xa6b5046 2024-09-16T11:13:23.181 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:23.181 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:23.187 INFO:teuthology.orchestra.run.smithi162.stdout:263 writesame 0xda56228 thru 0xda5f3f9 data_size 0x5(0x91d2 bytes) 2024-09-16T11:13:23.192 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:23.207 INFO:teuthology.orchestra.run.smithi162.stdout:264 trunc from 0xda5f3fa to 0x3570776 2024-09-16T11:13:23.233 INFO:teuthology.orchestra.run.smithi162.stdout:266 punch from 0x2d4c099 to 0x2d510a0, (0x5007 bytes) 2024-09-16T11:13:23.233 INFO:teuthology.orchestra.run.smithi162.stdout:268 read 0x1bd6872 thru 0x1be0f84 (0xa713 bytes) 2024-09-16T11:13:23.237 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:23.233+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:23.261 INFO:teuthology.orchestra.run.smithi162.stdout:269 write 0xb2500c4 thru 0xb2506e3 (0x620 bytes) 2024-09-16T11:13:23.268 INFO:teuthology.orchestra.run.smithi162.stdout:270 read 0x8ab01a1 thru 0x8ab085d (0x6bd bytes) 2024-09-16T11:13:23.268 INFO:teuthology.orchestra.run.smithi162.stdout:271 punch from 0x13674b9 to 0x1371bcd, (0xa714 bytes) 2024-09-16T11:13:23.268 INFO:teuthology.orchestra.run.smithi162.stdout:274 writesame 0xa447aab thru 0xa44b241 data_size 0x13(0x3797 bytes) 2024-09-16T11:13:23.273 INFO:teuthology.orchestra.run.smithi162.stdout:275 read 0x50423b3 thru 0x5050d3b (0xe989 bytes) 2024-09-16T11:13:23.283 INFO:teuthology.orchestra.run.smithi162.stdout:276 write 0xb5e14da thru 0xb5e93ad (0x7ed4 bytes) 2024-09-16T11:13:23.289 INFO:teuthology.orchestra.run.smithi162.stdout:277 read 0x3908d8e thru 0x390910c (0x37f bytes) 2024-09-16T11:13:23.289 INFO:teuthology.orchestra.run.smithi162.stdout:279 writesame 0x34dcad9 thru 0x34e31e9 data_size 0x5(0x6711 bytes) 2024-09-16T11:13:23.298 INFO:teuthology.orchestra.run.smithi162.stdout:280 write 0x2a549fc thru 0x2a5ab8b (0x6190 bytes) 2024-09-16T11:13:23.300 INFO:teuthology.orchestra.run.smithi162.stdout:281 read 0x6892543 thru 0x68965c1 (0x407f bytes) 2024-09-16T11:13:23.300 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:23.301 INFO:teuthology.orchestra.run.smithi162.stdout:283 write 0x5589dd4 thru 0x558c5de (0x280b bytes) 2024-09-16T11:13:23.306 INFO:teuthology.orchestra.run.smithi162.stdout:285 writesame 0xb9f84e thru 0xbae999 data_size 0x2(0xf14c bytes) 2024-09-16T11:13:23.337 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:23.333+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:23.338 INFO:teuthology.orchestra.run.smithi162.stdout:288 punch from 0x2131961 to 0x21380c1, (0x6760 bytes) 2024-09-16T11:13:23.338 INFO:teuthology.orchestra.run.smithi162.stdout:289 punch from 0x253ab7b to 0x2548774, (0xdbf9 bytes) 2024-09-16T11:13:23.338 INFO:teuthology.orchestra.run.smithi162.stdout:290 read 0x50d5987 thru 0x50e1b6e (0xc1e8 bytes) 2024-09-16T11:13:23.338 INFO:teuthology.orchestra.run.smithi162.stdout:291 writesame 0x3708c1b thru 0x370d078 data_size 0x3(0x445e bytes) 2024-09-16T11:13:23.341 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:23.341 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:23.349 INFO:teuthology.orchestra.run.smithi162.stdout:292 read 0x8a3fb7d thru 0x8a47446 (0x78ca bytes) 2024-09-16T11:13:23.349 INFO:teuthology.orchestra.run.smithi162.stdout:293 read 0x296ea49 thru 0x297b1ca (0xc782 bytes) 2024-09-16T11:13:23.349 INFO:teuthology.orchestra.run.smithi162.stdout:294 punch from 0x2784415 to 0x278c56b, (0x8156 bytes) 2024-09-16T11:13:23.349 INFO:teuthology.orchestra.run.smithi162.stdout:295 writesame 0x4d3d07f thru 0x4d4630f data_size 0xb(0x9291 bytes) 2024-09-16T11:13:23.350 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:23.351 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:23.373 INFO:teuthology.orchestra.run.smithi162.stdout:296 trunc from 0xb5e93ae to 0x3967b17 2024-09-16T11:13:23.401 INFO:teuthology.orchestra.run.smithi162.stdout:297 clone 4 order 21 su 262144 sc 9 2024-09-16T11:13:23.438 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:23.433+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:23.445 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:23.445 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:23.453 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:23.474 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 3.13636 -> 3.13636 (expected baseline 3.3197) 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.5 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 19 -> 19 pgs (1.47778 -> 1.47778 * mean) 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 0.05 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0032 2024-09-16T11:13:23.475 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0316 2024-09-16T11:13:23.477 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:13:23.484 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:23.493 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:13:23.493 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-16T11:13:23.503 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:23.524 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:23.539 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:23.533+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:23.542 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:23.564 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:23.586 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:23.604 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:23.604 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:23.610 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:23.613 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:23.640 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:23.633+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:23.643 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:23.645 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:23.646 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:23.711 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:23.712 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:23.721 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:23.740 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:23.737+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:23.795 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:13:23.795 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:13:23.795 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 3.13636 -> 3.13636 (expected baseline 3.3197) 2024-09-16T11:13:23.795 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.5 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:13:23.795 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 19 -> 19 pgs (1.47778 -> 1.47778 * mean) 2024-09-16T11:13:23.795 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:23.796 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:13:23.796 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:13:23.796 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:13:23.796 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0032 2024-09-16T11:13:23.796 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0316 2024-09-16T11:13:23.799 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:13:23.809 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:13:23.809 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.6846161590000008, 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066095169999999996, 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 11 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130364+0000", 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2200000000000001e-06 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130378+0000", 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3498e-05 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130415+0000", 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6928999999999998e-05 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130431+0000", 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5834000000000001e-05 2024-09-16T11:13:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130591+0000", 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016076400000000001 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130610+0000", 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8244999999999999e-05 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.133935+0000", 2024-09-16T11:13:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033251650000000002 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134174+0000", 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023965400000000001 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134553+0000", 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037902299999999998 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136098+0000", 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0015447010000000001 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136814+0000", 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00071608299999999995 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136888+0000", 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3526e-05 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136971+0000", 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2874999999999995e-05 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:456 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [read 0~40] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.158798+0000", 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.656179302, 2024-09-16T11:13:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037971150000000002, 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 456 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158798+0000", 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158798+0000", 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158801+0000", 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9019999999999999e-06 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158812+0000", 2024-09-16T11:13:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1301000000000001e-05 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158853+0000", 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1032000000000003e-05 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.158869+0000", 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5556000000000001e-05 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.159009+0000", 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014004999999999999 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.159036+0000", 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7577999999999999e-05 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.162236+0000", 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031991789999999999 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.162312+0000", 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6402000000000005e-05 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.162595+0000", 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028311500000000002 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:537 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.285597+0000", 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5293803710000002, 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035981170000000001, 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 537 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285597+0000", 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285597+0000", 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285600+0000", 2024-09-16T11:13:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7499999999999999e-06 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285613+0000", 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.362e-05 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285653+0000", 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9688999999999999e-05 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285668+0000", 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5034e-05 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285797+0000", 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128582 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.285968+0000", 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000171483 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.286616+0000", 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00064846299999999995 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.288322+0000", 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017059849999999999 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.288915+0000", 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00059270999999999998 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.289084+0000", 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016872999999999999 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.289195+0000", 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011107099999999999 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:544 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5176016519999997, 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073024969999999998, 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 544 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297378+0000", 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.99e-06 2024-09-16T11:13:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297388+0000", 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0407e-05 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297415+0000", 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7498e-05 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297426+0000", 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0957e-05 2024-09-16T11:13:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297594+0000", 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167157 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297849+0000", 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025576 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.298833+0000", 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098353599999999992 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300723+0000", 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001890395 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304585+0000", 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038620709999999999 2024-09-16T11:13:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304639+0000", 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3193999999999999e-05 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304678+0000", 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9532e-05 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:549 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.517173278, 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007009576, 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 549 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297806+0000", 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.747e-06 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297814+0000", 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2900000000000002e-06 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297858+0000", 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4295000000000002e-05 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297868+0000", 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2639999999999997e-06 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299859+0000", 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019912609999999998 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299944+0000", 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5069999999999997e-05 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300312+0000", 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036818899999999999 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304727+0000", 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044150689999999998 2024-09-16T11:13:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304734+0000", 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304779+0000", 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5055000000000001e-05 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304814+0000", 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4770000000000001e-05 2024-09-16T11:13:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:591 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5143794900000001, 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043393090000000004, 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 591 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300600+0000", 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1320000000000001e-06 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300611+0000", 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0961000000000001e-05 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300633+0000", 2024-09-16T11:13:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1999000000000001e-05 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300644+0000", 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1632e-05 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300770+0000", 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012577899999999999 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300927+0000", 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000157186 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301666+0000", 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073808200000000004 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304821+0000", 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031557349999999998 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304859+0000", 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7450999999999998e-05 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304902+0000", 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3579999999999999e-05 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304937+0000", 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4771999999999998e-05 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:592 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5140986500000002, 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061751929999999998, 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 592 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300880+0000", 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6080000000000001e-06 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300890+0000", 2024-09-16T11:13:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4040000000000001e-06 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300907+0000", 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6903000000000001e-05 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300916+0000", 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5219999999999996e-06 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302514+0000", 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00159807 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302687+0000", 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000173155 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303339+0000", 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0006514 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306894+0000", 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035556720000000002 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306940+0000", 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5893999999999998e-05 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307005+0000", 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4432999999999998e-05 2024-09-16T11:13:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307054+0000", 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9132000000000003e-05 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:593 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5137934959999999, 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082575289999999996, 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 593 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301185+0000", 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2860000000000001e-06 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301193+0000", 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.678e-06 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301209+0000", 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5797000000000001e-05 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301218+0000", 2024-09-16T11:13:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1980000000000007e-06 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303958+0000", 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027405810000000002 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304015+0000", 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7074000000000003e-05 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304249+0000", 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023344199999999999 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307114+0000", 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028655569999999999 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309341+0000", 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022266339999999999 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309401+0000", 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0436000000000002e-05 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309441+0000", 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9845999999999998e-05 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:595 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5137425799999997, 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010351252, 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 595 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2999999999999999e-07 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301240+0000", 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8359999999999999e-06 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301269+0000", 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8459e-05 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301278+0000", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7949999999999996e-06 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304969+0000", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003691864 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305136+0000", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016632999999999999 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305576+0000", 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043990000000000001 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309262+0000", 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036865930000000002 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311524+0000", 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022620909999999999 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311559+0000", 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4597000000000001e-05 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311586+0000", 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6957e-05 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:597 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5136005969999999, 2024-09-16T11:13:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010265962, 2024-09-16T11:13:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 597 2024-09-16T11:13:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301378+0000", 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.657e-06 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301387+0000", 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3860000000000007e-06 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301402+0000", 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4875000000000001e-05 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301411+0000", 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5820000000000008e-06 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305997+0000", 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045854169999999996 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306078+0000", 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1614999999999997e-05 2024-09-16T11:13:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306441+0000", 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036263000000000002 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309329+0000", 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028879650000000002 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311596+0000", 2024-09-16T11:13:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022673939999999998 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311621+0000", 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4797e-05 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311643+0000", 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1644000000000001e-05 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:598 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5135169570000002, 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010230927000000001, 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 598 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301462+0000", 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301470+0000", 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6279999999999996e-06 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301484+0000", 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4314999999999999e-05 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301512+0000", 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8218999999999999e-05 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307142+0000", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056298169999999996 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307264+0000", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121704 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307701+0000", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043725099999999999 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311457+0000", 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037559 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311649+0000", 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019221900000000001 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311673+0000", 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3603000000000001e-05 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311691+0000", 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8609000000000001e-05 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:599 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5133056910000002, 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014841653999999999, 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 599 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301673+0000", 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3459999999999999e-06 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301681+0000", 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0949999999999996e-06 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301696+0000", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4932e-05 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301705+0000", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6009999999999995e-06 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308183+0000", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064783840000000002 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308260+0000", 2024-09-16T11:13:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6601999999999996e-05 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308708+0000", 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044877600000000001 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311516+0000", 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028079189999999999 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316392+0000", 2024-09-16T11:13:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004875735 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316467+0000", 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4571000000000001e-05 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316513+0000", 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6693000000000002e-05 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:601 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5128393949999999, 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014436584000000001, 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 601 2024-09-16T11:13:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302139+0000", 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302152+0000", 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3608000000000001e-05 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302163+0000", 2024-09-16T11:13:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1012000000000001e-05 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302169+0000", 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1309999999999997e-06 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309509+0000", 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073404869999999997 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309603+0000", 2024-09-16T11:13:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4374000000000001e-05 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310124+0000", 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052012300000000005 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312552+0000", 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024287969999999999 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316527+0000", 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039741569999999999 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316553+0000", 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945999999999999e-05 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316574+0000", 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1937999999999999e-05 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:603 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.512153955, 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019179324000000001, 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 603 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302824+0000", 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.06e-06 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302830+0000", 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9209999999999997e-06 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302844+0000", 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3304e-05 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302850+0000", 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310529+0000", 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076792980000000002 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310601+0000", 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1070999999999998e-05 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311059+0000", 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045884099999999998 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313207+0000", 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002147805 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321862+0000", 2024-09-16T11:13:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086545330000000007 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321919+0000", 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7130999999999999e-05 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322003+0000", 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3794000000000004e-05 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:604 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5113479590000001, 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.018458927999999999, 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 604 2024-09-16T11:13:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303631+0000", 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5579999999999999e-06 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303638+0000", 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303654+0000", 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6628000000000001e-05 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303662+0000", 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8390000000000007e-06 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311727+0000", 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080650300000000008 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311786+0000", 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8507000000000001e-05 2024-09-16T11:13:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312144+0000", 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358821 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.314152+0000", 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002008045 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322013+0000", 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078606920000000007 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322053+0000", 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322088+0000", 2024-09-16T11:13:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4972000000000003e-05 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:605 4.13 e38/31 4:cae7a2e7:::rbd_data.11b52605a025.0000000000000001:head v 38'7, mlcod=35'4)", 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5094160919999999, 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056889410000000003, 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305562+0000", 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305567+0000", 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5949999999999998e-06 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305577+0000", 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305584+0000", 2024-09-16T11:13:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3670000000000002e-06 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305659+0000", 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5593999999999995e-05 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305672+0000", 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3193000000000001e-05 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311201+0000", 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055283249999999997 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311250+0000", 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9382000000000003e-05 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:613 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'8, mlcod=37'5)", 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5024485730000001, 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095140820000000001, 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312549+0000", 2024-09-16T11:13:23.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9432999999999998e-05 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312559+0000", 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0455000000000001e-05 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312565+0000", 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4609999999999999e-06 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312594+0000", 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9456000000000001e-05 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312622+0000", 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7586999999999999e-05 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322004+0000", 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093818349999999998 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322043+0000", 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9167e-05 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.924 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:614 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'10, mlcod=37'5)", 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.5019575619999999, 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010261963000000001, 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313021+0000", 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5799999999999998e-07 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313028+0000", 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1400000000000002e-06 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313041+0000", 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2958e-05 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313047+0000", 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0530000000000003e-06 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313090+0000", 2024-09-16T11:13:23.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2936999999999999e-05 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313105+0000", 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5199e-05 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323210+0000", 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010104958000000001 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323282+0000", 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1860000000000007e-05 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:617 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.4981974259999999, 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060101130000000001, 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 617 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316781+0000", 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.279e-06 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316910+0000", 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012939700000000001 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316922+0000", 2024-09-16T11:13:23.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1894e-05 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316928+0000", 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5559999999999998e-06 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316974+0000", 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5587000000000002e-05 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317094+0000", 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000120586 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317242+0000", 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147517 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322697+0000", 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054555960000000001 2024-09-16T11:13:23.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322708+0000", 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0805000000000001e-05 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322745+0000", 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7280000000000002e-05 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322790+0000", 2024-09-16T11:13:23.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4616000000000001e-05 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:618 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.4980355699999999, 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006757379, 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 618 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316943+0000", 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9199999999999999e-07 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316945+0000", 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5009999999999999e-06 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:23.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316952+0000", 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5980000000000002e-06 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316955+0000", 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4879999999999999e-06 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318095+0000", 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011395050000000001 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318151+0000", 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6577999999999999e-05 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318284+0000", 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013229499999999999 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:23.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323278+0000", 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049947200000000002 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323605+0000", 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032718800000000001 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323658+0000", 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2961999999999999e-05 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:23.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323699+0000", 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0652000000000003e-05 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:23.941 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:23.943 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:23.943 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:23.937+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:24.044 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:24.037+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:24.093 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:24.093 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:24.093 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:24.093 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 3.13636 -> 3.13636 (expected baseline 3.3197) 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.5 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 19 -> 19 pgs (1.47778 -> 1.47778 * mean) 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 0.05 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0032 2024-09-16T11:13:24.100 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0317 2024-09-16T11:13:24.103 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:24.103 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:13:24.113 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:13:24.114 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-16T11:13:24.157 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:24.153+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:24.184 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:13:24.216 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:24.216 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:24.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:24.245 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:24.241+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:24.310 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone3 from 0x3967b17 (overlap 0x1c6be41) to 0x13e51fa 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.330 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.548408342999998, 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:24.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:24.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:24.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.160482643000002, 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:24.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:24.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.160331313, 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:24.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.160279826999997, 2024-09-16T11:13:24.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:24.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:24.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.160209678999998, 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:24.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:24.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:24.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.160037209000002, 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:24.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.366 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.159978887000001, 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:24.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:24.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.159826766999998, 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:24.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.159648248000003, 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:24.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:24.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.159615275, 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:24.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:24.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:24.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.607449783, 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.390 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:24.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:24.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:24.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.607427549999997, 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:24.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:24.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.401 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.607298753000002, 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:24.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:24.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:24.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.607240408000003, 2024-09-16T11:13:24.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:24.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:24.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.607186018999997, 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:24.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.607051712000001, 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.419 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:24.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:24.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.825954191999998, 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:24.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:24.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:24.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.825896342999997, 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:24.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:24.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.436 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.825809030999999, 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:24.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:24.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:24.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.825570902999999, 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:24.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:24.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:24.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:24.448 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:24.449 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:24.450 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:24.451 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:24.445+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:24.451 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:13:24.451 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:13:24.451 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 3.13636 -> 3.13636 (expected baseline 3.3197) 2024-09-16T11:13:24.451 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.5 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:13:24.451 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 19 -> 19 pgs (1.47778 -> 1.47778 * mean) 2024-09-16T11:13:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:24.452 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:13:24.452 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:13:24.452 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:13:24.452 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0032 2024-09-16T11:13:24.452 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0317 2024-09-16T11:13:24.454 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:13:24.463 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:13:24.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-16T11:13:24.500 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T11:13:24.516 INFO:teuthology.orchestra.run.smithi162.stdout:298 punch from 0x2f30579 to 0x2f39561, (0x8fe8 bytes) 2024-09-16T11:13:24.523 INFO:teuthology.orchestra.run.smithi162.stdout:299 punch from 0x22ee48f to 0x22fb59e, (0xd10f bytes) 2024-09-16T11:13:24.523 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.523 INFO:teuthology.orchestra.run.smithi162.stdout:301 punch from 0x3489c53 to 0x348f34c, (0x56f9 bytes) 2024-09-16T11:13:24.523 INFO:teuthology.orchestra.run.smithi162.stdout:302 flatten 2024-09-16T11:13:24.551 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:24.545+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:24.560 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:24.561 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:24.561 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:24.563 INFO:teuthology.orchestra.run.smithi162.stdout:303 punch from 0x113b353 to 0x1143e13, (0x8ac0 bytes) 2024-09-16T11:13:24.563 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.563 INFO:teuthology.orchestra.run.smithi162.stdout:305 read 0x12f145c thru 0x12f267c (0x1221 bytes) 2024-09-16T11:13:24.563 INFO:teuthology.orchestra.run.smithi162.stdout:306 read 0x148cae1 thru 0x1492c94 (0x61b4 bytes) 2024-09-16T11:13:24.563 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.573 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:24.589 INFO:teuthology.orchestra.run.smithi162.stdout:308 write 0xd0f3009 thru 0xd0f6aa5 (0x3a9d bytes) 2024-09-16T11:13:24.594 INFO:teuthology.orchestra.run.smithi162.stdout:309 read 0x636f990 thru 0x6370b89 (0x11fa bytes) 2024-09-16T11:13:24.596 INFO:teuthology.orchestra.run.smithi162.stdout:310 trunc from 0xd0f6aa6 to 0xbea3730 2024-09-16T11:13:24.619 INFO:teuthology.orchestra.run.smithi162.stdout:312 trunc from 0xbea3730 to 0xb9a73ff 2024-09-16T11:13:24.629 INFO:teuthology.orchestra.run.smithi162.stdout:314 write 0x9063a61 thru 0x906467f (0xc1f bytes) 2024-09-16T11:13:24.635 INFO:teuthology.orchestra.run.smithi162.stdout:316 read 0xa3c76af thru 0xa3d44d4 (0xce26 bytes) 2024-09-16T11:13:24.635 INFO:teuthology.orchestra.run.smithi162.stdout:317 writesame 0xafe080 thru 0xb0a910 data_size 0x69(0xc891 bytes) 2024-09-16T11:13:24.651 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:24.649+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:24.652 INFO:teuthology.orchestra.run.smithi162.stdout:318 trunc from 0xb9a73ff to 0x6c84e62 2024-09-16T11:13:24.673 INFO:teuthology.orchestra.run.smithi162.stdout:321 punch from 0x20a501c to 0x20b4a16, (0xf9fa bytes) 2024-09-16T11:13:24.673 INFO:teuthology.orchestra.run.smithi162.stdout:322 read 0x4cf093 thru 0x4d4dd8 (0x5d46 bytes) 2024-09-16T11:13:24.674 INFO:teuthology.orchestra.run.smithi162.stdout:323 write 0x4470793 thru 0x4470c57 (0x4c5 bytes) 2024-09-16T11:13:24.690 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:24.690 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:24.691 INFO:teuthology.orchestra.run.smithi162.stdout:325 writesame 0x6d3feb4 thru 0x6d4475c data_size 0x13(0x48a9 bytes) 2024-09-16T11:13:24.691 INFO:teuthology.orchestra.run.smithi162.stdout:326 read 0x2cd81ef thru 0x2cdf12f (0x6f41 bytes) 2024-09-16T11:13:24.691 INFO:teuthology.orchestra.run.smithi162.stdout:327 write 0x57f481d thru 0x57f62a8 (0x1a8c bytes) 2024-09-16T11:13:24.707 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:24.722 INFO:teuthology.orchestra.run.smithi162.stdout:328 write 0xa7b4685 thru 0xa7bafb2 (0x692e bytes) 2024-09-16T11:13:24.730 INFO:teuthology.orchestra.run.smithi162.stdout:329 read 0x9b6f23b thru 0x9b76216 (0x6fdc bytes) 2024-09-16T11:13:24.743 INFO:teuthology.orchestra.run.smithi162.stdout:330 trunc from 0xa7bafb3 to 0x32b503e 2024-09-16T11:13:24.751 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:24.745+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:24.786 INFO:teuthology.orchestra.run.smithi162.stdout:331 write 0x4f1b9b9 thru 0x4f22604 (0x6c4c bytes) 2024-09-16T11:13:24.792 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.792 INFO:teuthology.orchestra.run.smithi162.stdout:336 read 0x33d8945 thru 0x33dade8 (0x24a4 bytes) 2024-09-16T11:13:24.805 INFO:teuthology.orchestra.run.smithi162.stdout:337 write 0x56d1d8f thru 0x56d9455 (0x76c7 bytes) 2024-09-16T11:13:24.811 INFO:teuthology.orchestra.run.smithi162.stdout:342 read 0xe0e5c0 thru 0xe1396f (0x53b0 bytes) 2024-09-16T11:13:24.811 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.811 INFO:teuthology.orchestra.run.smithi162.stdout:344 punch from 0x5633ebc to 0x5637d4d, (0x3e91 bytes) 2024-09-16T11:13:24.814 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:13:24.814 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:13:24.814 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 3.13636 -> 3.13636 (expected baseline 3.3197) 2024-09-16T11:13:24.814 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.5 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 19 -> 19 pgs (1.47778 -> 1.47778 * mean) 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 0.05 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0032 2024-09-16T11:13:24.815 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0035 2024-09-16T11:13:24.818 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:13:24.827 INFO:teuthology.orchestra.run.smithi162.stdout:346 write 0x8694d10 thru 0x8695747 (0xa38 bytes) 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:44 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'26, mlcod=16'22)", 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.173890+0000", 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.657423623, 2024-09-16T11:13:24.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048216300000000004, 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173890+0000", 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173890+0000", 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173891+0000", 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.232e-06 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173973+0000", 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1514999999999995e-05 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.173993+0000", 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9677000000000002e-05 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174001+0000", 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2309999999999996e-06 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174084+0000", 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3512999999999998e-05 2024-09-16T11:13:24.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174118+0000", 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3312999999999998e-05 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.178647+0000", 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045288799999999999 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.178712+0000", 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5269000000000001e-05 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:45 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'27, mlcod=16'22)", 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.174640+0000", 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.656674252999998, 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044158899999999996, 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.841 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174640+0000", 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174640+0000", 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174641+0000", 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.133e-06 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174652+0000", 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1197e-05 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174671+0000", 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9434999999999999e-05 2024-09-16T11:13:24.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.174682+0000", 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1197e-05 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175186+0000", 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050357199999999998 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175214+0000", 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8200000000000001e-05 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179004+0000", 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037893470000000002 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179055+0000", 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1808999999999998e-05 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:46 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'28, mlcod=16'22)", 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.175230+0000", 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.656084276000001, 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048366809999999998, 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175230+0000", 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175230+0000", 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175231+0000", 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0470000000000001e-06 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175240+0000", 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4110000000000002e-06 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175254+0000", 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4064e-05 2024-09-16T11:13:24.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175267+0000", 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2526e-05 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175839+0000", 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057268499999999997 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175878+0000", 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8671999999999998e-05 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180028+0000", 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041503729999999997 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180066+0000", 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7902999999999999e-05 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:47 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'29, mlcod=16'22)", 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.175934+0000", 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.655379328999999, 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041741260000000002, 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175934+0000", 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175934+0000", 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.175936+0000", 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2559999999999999e-06 2024-09-16T11:13:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176009+0000", 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3695000000000005e-05 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176023+0000", 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3551e-05 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176044+0000", 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1162000000000002e-05 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176544+0000", 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049992299999999999 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.176576+0000", 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1949999999999997e-05 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180074+0000", 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034984479999999999 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180109+0000", 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4140999999999997e-05 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:51 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'24)", 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.179765+0000", 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.651548603000002, 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003872014, 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179765+0000", 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179765+0000", 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179766+0000", 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179772+0000", 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4459999999999998e-06 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179793+0000", 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0095e-05 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.179800+0000", 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2590000000000003e-06 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180220+0000", 2024-09-16T11:13:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000419875 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.180278+0000", 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8659000000000002e-05 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.183591+0000", 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033125870000000001 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.183637+0000", 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6226000000000001e-05 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4203.0:69 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'47, mlcod=16'41)", 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:50.725683+0000", 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.105630986999998, 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039048329999999999, 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725683+0000", 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725683+0000", 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725684+0000", 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3570000000000001e-06 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725770+0000", 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6034999999999999e-05 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725808+0000", 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7863e-05 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.725822+0000", 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4049000000000001e-05 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.726047+0000", 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022463499999999999 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.726079+0000", 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2703000000000002e-05 2024-09-16T11:13:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.729518+0000", 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003439059 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:50.729588+0000", 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9132000000000002e-05 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:756 4.b 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0464455800000003, 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035717454000000003, 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 756 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784871+0000", 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.497e-06 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790900+0000", 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060292039999999998 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790939+0000", 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8844999999999998e-05 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790951+0000", 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2143999999999999e-05 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791017+0000", 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5969999999999993e-05 2024-09-16T11:13:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791210+0000", 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000192993 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791635+0000", 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042530200000000002 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.812994+0000", 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021359210999999999 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820448+0000", 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0074540709999999996 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820509+0000", 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0625999999999999e-05 2024-09-16T11:13:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820586+0000", 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6590999999999999e-05 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:840 4.b 4:d76b06b8:::rbd_data.11b52605a025.0000000000000005:5 [sparse-read 14155776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.918469+0000", 2024-09-16T11:13:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9128444570000003, 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043864100000000003, 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 840 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918469+0000", 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918469+0000", 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918471+0000", 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9230000000000001e-06 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918644+0000", 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017256899999999999 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.922087+0000", 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034430490000000001 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.922107+0000", 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9709999999999999e-05 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.922406+0000", 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029963599999999999 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.922628+0000", 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022222499999999999 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.922856+0000", 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022729799999999999 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:873 4.b 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head [stat,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.924473+0000", 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9068403890000001, 2024-09-16T11:13:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005284731, 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 873 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924473+0000", 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924473+0000", 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924474+0000", 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1299999999999999e-07 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924478+0000", 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1219999999999997e-06 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924488+0000", 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0084e-05 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924494+0000", 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5539999999999997e-06 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.925028+0000", 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053452999999999997 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.925153+0000", 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124838 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.925445+0000", 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000291429 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.929223+0000", 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037782929999999998 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.929658+0000", 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00043514600000000002 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.929719+0000", 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1265000000000004e-05 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.929758+0000", 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8757000000000002e-05 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:923 4.b 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.994348+0000", 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.836966152, 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077816209999999999, 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 923 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994348+0000", 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994348+0000", 2024-09-16T11:13:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994352+0000", 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8789999999999996e-06 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994367+0000", 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5603e-05 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994415+0000", 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.829e-05 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994435+0000", 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9414000000000001e-05 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994644+0000", 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020923299999999999 2024-09-16T11:13:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.994828+0000", 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018363200000000001 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995074+0000", 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024654499999999998 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:21.000028+0000", 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049535100000000004 2024-09-16T11:13:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:21.002058+0000", 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020303669999999999 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:21.002100+0000", 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1609000000000001e-05 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:21.002129+0000", 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9539e-05 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1783 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.273587+0000", 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.55772635699999995, 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041621929999999998, 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1783 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273587+0000", 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273587+0000", 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273591+0000", 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2710000000000001e-06 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273605+0000", 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4748e-05 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273634+0000", 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8328999999999999e-05 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273646+0000", 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.202e-05 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273711+0000", 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4997000000000001e-05 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.273717+0000", 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.275740+0000", 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020231170000000001 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.275854+0000", 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011401899999999999 2024-09-16T11:13:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.276004+0000", 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014951499999999999 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.276659+0000", 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00065580399999999996 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.277683+0000", 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001023609 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.277722+0000", 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9271999999999999e-05 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.277750+0000", 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.73e-05 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1827 4.16 4:6eae1647:::rbd_data.11b51c0df580.0000000000000006:head [stat,truncate 1048576] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.332268+0000", 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.499046194, 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050853499999999998, 2024-09-16T11:13:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1827 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332268+0000", 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332268+0000", 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332270+0000", 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5909999999999998e-06 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332278+0000", 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9100000000000005e-06 2024-09-16T11:13:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332310+0000", 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1597000000000003e-05 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332319+0000", 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.02e-06 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332403+0000", 2024-09-16T11:13:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4578000000000002e-05 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332484+0000", 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0346999999999995e-05 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332633+0000", 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000149665 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.334583+0000", 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001949959 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337227+0000", 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002643631 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337295+0000", 2024-09-16T11:13:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7614999999999996e-05 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337353+0000", 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8437000000000003e-05 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2227 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.537629+0000", 2024-09-16T11:13:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.29368498799999998, 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069024350000000002, 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2227 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537629+0000", 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537629+0000", 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537629+0000", 2024-09-16T11:13:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537632+0000", 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4099999999999998e-06 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537635+0000", 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537637+0000", 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537675+0000", 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7894e-05 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.537750+0000", 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5203999999999994e-05 2024-09-16T11:13:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.538070+0000", 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031976200000000002 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.539902+0000", 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018322169999999999 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.544432+0000", 2024-09-16T11:13:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045291699999999999 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.544495+0000", 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3788000000000007e-05 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.544531+0000", 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5848e-05 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2289 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.541456+0000", 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28985773399999998, 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064543719999999999, 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2289 2024-09-16T11:13:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541456+0000", 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541456+0000", 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541458+0000", 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.463e-06 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541465+0000", 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4499999999999998e-06 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541484+0000", 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8672e-05 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541490+0000", 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7499999999999997e-06 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541538+0000", 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7778999999999998e-05 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541634+0000", 2024-09-16T11:13:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5790000000000003e-05 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542041+0000", 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040726 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.543368+0000", 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001327218 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.547745+0000", 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043768000000000001 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.547830+0000", 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4943000000000003e-05 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.547910+0000", 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0247000000000006e-05 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2299 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.541532+0000", 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28978194200000001, 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064984379999999996, 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2299 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541532+0000", 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541532+0000", 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541533+0000", 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0299999999999997e-07 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541537+0000", 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5090000000000004e-06 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541546+0000", 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1670000000000005e-06 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.541552+0000", 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8329999999999999e-06 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542655+0000", 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011023199999999999 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542730+0000", 2024-09-16T11:13:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5927000000000007e-05 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.543065+0000", 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033413699999999998 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.544251+0000", 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001186758 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.547957+0000", 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003705954 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.547997+0000", 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9437000000000002e-05 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.548030+0000", 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3492999999999999e-05 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2310 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.542540+0000", 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.288773891, 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060789579999999998, 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2310 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542540+0000", 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542540+0000", 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542541+0000", 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3880000000000001e-06 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542548+0000", 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542562+0000", 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309e-05 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.542569+0000", 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.618e-06 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.543399+0000", 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00083063500000000003 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.543481+0000", 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1174000000000006e-05 2024-09-16T11:13:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.543842+0000", 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036151199999999998 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.547441+0000", 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035986220000000001 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.548516+0000", 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010750390000000001 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.548582+0000", 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6483999999999996e-05 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.548619+0000", 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6628000000000002e-05 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2337 4.6 4:679a8f61:::rbd_data.11b5220ac4a2.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.545967+0000", 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.28534706500000001, 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044750959999999996, 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2337 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.545967+0000", 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.545967+0000", 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.545968+0000", 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1060000000000001e-06 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.546342+0000", 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037461699999999999 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.546356+0000", 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3315999999999999e-05 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.546362+0000", 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3210000000000001e-06 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.546415+0000", 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3418999999999998e-05 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.546545+0000", 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012908200000000001 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.546797+0000", 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025243900000000003 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.548319+0000", 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015223229999999999 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.550331+0000", 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020117339999999998 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.550401+0000", 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0298999999999999e-05 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.550442+0000", 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0439999999999999e-05 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2421 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.673745+0000", 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.157568336, 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0037359939999999999, 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2421 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673745+0000", 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673745+0000", 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673748+0000", 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.063e-06 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673758+0000", 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0009e-05 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673781+0000", 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.376e-05 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673791+0000", 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7100000000000002e-06 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.673910+0000", 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011938999999999999 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.674141+0000", 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000230886 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.675207+0000", 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010656909999999999 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.676845+0000", 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001637913 2024-09-16T11:13:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.677383+0000", 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053850000000000002 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.677438+0000", 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4761e-05 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.677481+0000", 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3310999999999999e-05 2024-09-16T11:13:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2432 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.691034+0000", 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.14027983299999999, 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047843460000000001, 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2432 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691034+0000", 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691034+0000", 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691035+0000", 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4330000000000001e-06 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691041+0000", 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.066e-06 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691055+0000", 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4049000000000001e-05 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691063+0000", 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6750000000000002e-06 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691139+0000", 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5630999999999995e-05 2024-09-16T11:13:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691242+0000", 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102935 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.691665+0000", 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042304099999999998 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.693626+0000", 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019607380000000001 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.695524+0000", 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018984780000000001 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.695703+0000", 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017949599999999999 2024-09-16T11:13:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.695818+0000", 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011480399999999999 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2443 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.721724+0000", 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.109590167, 2024-09-16T11:13:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045081790000000002, 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2443 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721724+0000", 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721724+0000", 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721725+0000", 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.61e-06 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721735+0000", 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0198e-05 2024-09-16T11:13:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721757+0000", 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1498e-05 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721765+0000", 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4200000000000007e-06 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721836+0000", 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0499000000000004e-05 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.721964+0000", 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012814400000000001 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.722474+0000", 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050981699999999997 2024-09-16T11:13:24.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.724722+0000", 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002248103 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.726062+0000", 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013400090000000001 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.726156+0000", 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4341999999999999e-05 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.726232+0000", 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5538999999999996e-05 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:24.951 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:24.953 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:347 write 0x882c2f8 thru 0x8833ad7 (0x77e0 bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:348 writesame 0x49983da thru 0x4999293 data_size 0x1d(0xeba bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:353 write 0x497017e thru 0x4975ca3 (0x5b26 bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:354 read 0x6308292 thru 0x630cc66 (0x49d5 bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:356 writesame 0xd7eb926 thru 0xd7ecbe7 data_size 0xe(0x12c2 bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:358 trunc from 0xd7ecbe8 to 0x9b91b0e 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:359 write 0x49144a0 thru 0x49217f9 (0xd35a bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:362 write 0x5bfc002 thru 0x5c018e0 (0x58df bytes) 2024-09-16T11:13:24.955 INFO:teuthology.orchestra.run.smithi162.stdout:363 writesame 0x3e99c77 thru 0x3ea0c7e data_size 0x3c(0x7008 bytes) 2024-09-16T11:13:24.956 INFO:teuthology.orchestra.run.smithi162.stdout:364 writesame 0xabaf3b9 thru 0xabb9bbc data_size 0x2(0xa804 bytes) 2024-09-16T11:13:24.956 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:24.949+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:24.962 INFO:teuthology.orchestra.run.smithi162.stdout:367 trunc from 0xabb9bbd to 0x20f2d66 2024-09-16T11:13:24.996 INFO:teuthology.orchestra.run.smithi162.stdout:368 writesame 0x224be52 thru 0x224c163 data_size 0x3(0x312 bytes) 2024-09-16T11:13:24.996 INFO:teuthology.orchestra.run.smithi162.stdout:369 punch from 0x191cd8e to 0x192c3b9, (0xf62b bytes) 2024-09-16T11:13:24.997 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:24.997 INFO:teuthology.orchestra.run.smithi162.stdout:371 read 0x11d62e5 thru 0x11d86f2 (0x240e bytes) 2024-09-16T11:13:25.029 INFO:teuthology.orchestra.run.smithi162.stdout:373 writesame 0xd2b4136 thru 0xd2c2fce data_size 0x1(0xee99 bytes) 2024-09-16T11:13:25.036 INFO:teuthology.orchestra.run.smithi162.stdout:375 trunc from 0xd2c2fcf to 0x8a8da70 2024-09-16T11:13:25.053 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:25.049+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:25.061 INFO:teuthology.orchestra.run.smithi162.stdout:377 trunc from 0x8a8da70 to 0x2f98318 2024-09-16T11:13:25.067 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:25.067 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:25.067 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:25.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:25.088 INFO:teuthology.orchestra.run.smithi162.stdout:378 trunc from 0x2f98318 to 0xba61d3d 2024-09-16T11:13:25.101 INFO:teuthology.orchestra.run.smithi162.stdout:379 punch from 0x906980 to 0x90b5ff, (0x4c7f bytes) 2024-09-16T11:13:25.101 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.101 INFO:teuthology.orchestra.run.smithi162.stdout:381 write 0xa708d57 thru 0xa70b407 (0x26b1 bytes) 2024-09-16T11:13:25.119 INFO:teuthology.orchestra.run.smithi162.stdout:382 write 0xc873c2e thru 0xc876bb3 (0x2f86 bytes) 2024-09-16T11:13:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:383 punch from 0x24cdf0 to 0x259e7c, (0xd08c bytes) 2024-09-16T11:13:25.142 INFO:teuthology.orchestra.run.smithi162.stdout:384 trunc from 0xc876bb4 to 0x418d866 2024-09-16T11:13:25.154 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:25.149+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:25.162 INFO:teuthology.orchestra.run.smithi162.stdout:386 read 0x372593f thru 0x3728479 (0x2b3b bytes) 2024-09-16T11:13:25.162 INFO:teuthology.orchestra.run.smithi162.stdout:387 write 0x7312aa thru 0x7408c5 (0xf61c bytes) 2024-09-16T11:13:25.189 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:13:25.198 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:25.198 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:25.199 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:25.199 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:25.201 INFO:teuthology.orchestra.run.smithi162.stdout:389 writesame 0xe93ba46 thru 0xe947ec8 data_size 0x29(0xc483 bytes) 2024-09-16T11:13:25.201 INFO:teuthology.orchestra.run.smithi162.stdout:390 read 0x5180886 thru 0x5181c32 (0x13ad bytes) 2024-09-16T11:13:25.201 INFO:teuthology.orchestra.run.smithi162.stdout:391 writesame 0x5a304d4 thru 0x5a37662 data_size 0x1(0x718f bytes) 2024-09-16T11:13:25.208 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:25.221 INFO:teuthology.orchestra.run.smithi162.stdout:393 read 0x696ef0b thru 0x69747f0 (0x58e6 bytes) 2024-09-16T11:13:25.233 INFO:teuthology.orchestra.run.smithi162.stdout:395 trunc from 0xe947ec9 to 0x67b0b0c 2024-09-16T11:13:25.248 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.248 INFO:teuthology.orchestra.run.smithi162.stdout:397 write 0x3b8278e thru 0x3b84a9f (0x2312 bytes) 2024-09-16T11:13:25.254 INFO:teuthology.orchestra.run.smithi162.stdout:399 read 0x1c8b06c thru 0x1c8bbcd (0xb62 bytes) 2024-09-16T11:13:25.255 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:25.249+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:25.280 INFO:teuthology.orchestra.run.smithi162.stdout:400 writesame 0xc7c0d13 thru 0xc7d0296 data_size 0x26(0xf584 bytes) 2024-09-16T11:13:25.280 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.280 INFO:teuthology.orchestra.run.smithi162.stdout:403 write 0x8c3a5d3 thru 0x8c45e1d (0xb84b bytes) 2024-09-16T11:13:25.288 INFO:teuthology.orchestra.run.smithi162.stdout:404 write 0x56388da thru 0x5642b88 (0xa2af bytes) 2024-09-16T11:13:25.301 INFO:teuthology.orchestra.run.smithi162.stdout:405 read 0x7b8eb31 thru 0x7b9bcaa (0xd17a bytes) 2024-09-16T11:13:25.301 INFO:teuthology.orchestra.run.smithi162.stdout:407 read 0xc0f476a thru 0xc0f4a0d (0x2a4 bytes) 2024-09-16T11:13:25.301 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.301 INFO:teuthology.orchestra.run.smithi162.stdout:409 write 0x51aa9d1 thru 0x51ad1d6 (0x2806 bytes) 2024-09-16T11:13:25.303 INFO:teuthology.orchestra.run.smithi162.stdout:410 punch from 0x47a0ead to 0x47a213c, (0x128f bytes) 2024-09-16T11:13:25.304 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.304 INFO:teuthology.orchestra.run.smithi162.stdout:412 read 0xa6ca764 thru 0xa6d7472 (0xcd0f bytes) 2024-09-16T11:13:25.304 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.304 INFO:teuthology.orchestra.run.smithi162.stdout:415 writesame 0x98d4481 thru 0x98d5141 data_size 0x1(0xcc1 bytes) 2024-09-16T11:13:25.310 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.311 INFO:teuthology.orchestra.run.smithi162.stdout:417 write 0x72337fc thru 0x7237b42 (0x4347 bytes) 2024-09-16T11:13:25.317 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.317 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.317 INFO:teuthology.orchestra.run.smithi162.stdout:421 write 0x3492048 thru 0x3495d77 (0x3d30 bytes) 2024-09-16T11:13:25.323 INFO:teuthology.orchestra.run.smithi162.stdout:423 read 0xae74afb thru 0xae7fff5 (0xb4fb bytes) 2024-09-16T11:13:25.323 INFO:teuthology.orchestra.run.smithi162.stdout:425 write 0x3e90172 thru 0x3e9d5d6 (0xd465 bytes) 2024-09-16T11:13:25.330 INFO:teuthology.orchestra.run.smithi162.stdout:426 write 0xbfe7b3f thru 0xbff6a37 (0xeef9 bytes) 2024-09-16T11:13:25.332 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:25.332 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:25.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:25.332 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:544 4.a e38/31 4:57f43eeb:::rbd_object_map.11b52605a025:head v 38'14, mlcod=38'12)", 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.300645+0000", 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.0279275300000004, 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042862730000000002, 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300645+0000", 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300645+0000", 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300647+0000", 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.547e-06 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300688+0000", 2024-09-16T11:13:25.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1341999999999998e-05 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300775+0000", 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7471000000000003e-05 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300827+0000", 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1592999999999999e-05 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.301179+0000", 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035227199999999999 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.301251+0000", 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2025000000000002e-05 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.304804+0000", 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035531460000000001 2024-09-16T11:13:25.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.304931+0000", 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012687700000000001 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:756 4.b e39/31 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head v 39'5, mlcod=38'2)", 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.536189201, 2024-09-16T11:13:25.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027783187000000001, 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792386+0000", 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.667e-06 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799237+0000", 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006850737 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799272+0000", 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5147000000000001e-05 2024-09-16T11:13:25.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799284+0000", 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2391999999999999e-05 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799389+0000", 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104759 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799444+0000", 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5118000000000003e-05 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.819973+0000", 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020528497 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820167+0000", 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019387000000000001 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.341 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:828 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13441105~9082] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4713734560000002, 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042301801999999999, 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 828 2024-09-16T11:13:25.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857203+0000", 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299999999999999e-06 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862148+0000", 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049449400000000001 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862190+0000", 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2153000000000001e-05 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862203+0000", 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2604e-05 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862330+0000", 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012740900000000001 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862670+0000", 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033967399999999999 2024-09-16T11:13:25.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.863645+0000", 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00097586600000000004 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.884164+0000", 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020518723999999999 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899338+0000", 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015173623000000001 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899435+0000", 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7671999999999996e-05 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899501+0000", 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5407000000000006e-05 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.347 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:871 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [stat,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.918369+0000", 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4102031620000002, 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096987259999999995, 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 871 2024-09-16T11:13:25.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918369+0000", 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918369+0000", 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918373+0000", 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918392+0000", 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9326000000000001e-05 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918427+0000", 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4938999999999999e-05 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918444+0000", 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6534000000000001e-05 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918601+0000", 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000157186 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918929+0000", 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032847799999999999 2024-09-16T11:13:25.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919317+0000", 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038708900000000002 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924671+0000", 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053548659999999998 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.927967+0000", 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032950940000000001 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.928029+0000", 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2204000000000003e-05 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.928068+0000", 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9359e-05 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:874 4.f 4:f1087b7b:::rbd_data.11b585795d6b.0000000000000005:head [stat,zero 8388608~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.918502+0000", 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4100703880000003, 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004962574, 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 874 2024-09-16T11:13:25.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918502+0000", 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918502+0000", 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918504+0000", 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3519999999999999e-06 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918510+0000", 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6989999999999997e-06 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918534+0000", 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3957000000000001e-05 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918545+0000", 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0671e-05 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918740+0000", 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195236 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919007+0000", 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026729400000000001 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.923465+0000", 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044573649999999996 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:891 4.9 e39/31 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head v 39'1, mlcod=0'0)", 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.935936+0000", 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.3926366679999997, 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0097245609999999996, 2024-09-16T11:13:25.358 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.935936+0000", 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.935936+0000", 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.935938+0000", 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.933e-06 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939444+0000", 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035060769999999998 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939460+0000", 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.605e-05 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939467+0000", 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.046e-06 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939500+0000", 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3151999999999997e-05 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939529+0000", 2024-09-16T11:13:25.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9354999999999999e-05 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.945603+0000", 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060735329999999999 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.945660+0000", 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7414999999999998e-05 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:921 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.991351+0000", 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.3372213239999997, 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065742999999999999, 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 921 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991351+0000", 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991351+0000", 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991353+0000", 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1069999999999999e-06 2024-09-16T11:13:25.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991364+0000", 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0560000000000001e-05 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991398+0000", 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4001999999999997e-05 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991410+0000", 2024-09-16T11:13:25.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.217e-05 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991530+0000", 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120072 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991676+0000", 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000145725 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:25.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991841+0000", 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016474799999999999 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.996861+0000", 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050203219999999998 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.997546+0000", 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068512200000000003 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.997811+0000", 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026539000000000001 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.997926+0000", 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114082 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.368 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:920 4.9 e39/31 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head v 39'2, mlcod=0'0)", 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.991954+0000", 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.3366181340000001, 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041006439999999996, 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991954+0000", 2024-09-16T11:13:25.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991954+0000", 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991956+0000", 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991973+0000", 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6988000000000001e-05 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.992001+0000", 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7705e-05 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.992019+0000", 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8573999999999999e-05 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.992097+0000", 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7101999999999995e-05 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.992118+0000", 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1432000000000001e-05 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995937+0000", 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038190479999999998 2024-09-16T11:13:25.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.996055+0000", 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011806099999999999 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:923 4.b e39/31 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head v 39'8, mlcod=39'6)", 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.995274+0000", 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.3332986980000001, 2024-09-16T11:13:25.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045649009999999997, 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995274+0000", 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995274+0000", 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995275+0000", 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.282e-06 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995285+0000", 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7340000000000003e-06 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995315+0000", 2024-09-16T11:13:25.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9794e-05 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995325+0000", 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0509000000000001e-05 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995464+0000", 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013907899999999999 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995504+0000", 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9501e-05 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999779+0000", 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004275048 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999839+0000", 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9954000000000002e-05 2024-09-16T11:13:25.377 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1816 4.f 4:f1d27fab:::rbd_data.11b51c0df580.000000000000000a:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.312598+0000", 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.015974545, 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077270070000000001, 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.378 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1816 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312598+0000", 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312598+0000", 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312600+0000", 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2749999999999998e-06 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312610+0000", 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0039e-05 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312634+0000", 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3274000000000001e-05 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312648+0000", 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4163e-05 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312714+0000", 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5973e-05 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312793+0000", 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9536999999999994e-05 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312980+0000", 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000186319 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.314393+0000", 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001413032 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320230+0000", 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058371040000000001 2024-09-16T11:13:25.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320276+0000", 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6631999999999997e-05 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320325+0000", 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659000000000003e-05 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:1819 4.1d e41/31 4:ba69c8d3:::rbd_data.11b51c0df580.000000000000000f:head v 41'19, mlcod=40'14)", 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.315567+0000", 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.0130055499999999, 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041240449999999998, 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315567+0000", 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315567+0000", 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315568+0000", 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000002e-07 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315575+0000", 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3359999999999997e-06 2024-09-16T11:13:25.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315590+0000", 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4979e-05 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315597+0000", 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4500000000000001e-06 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315639+0000", 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2361000000000003e-05 2024-09-16T11:13:25.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315657+0000", 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7691999999999999e-05 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319601+0000", 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039445579999999999 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319691+0000", 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9808999999999993e-05 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1822 4.12 4:4fb601e4:::rbd_object_map.11b51c0df580:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.325376+0000", 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.0031966219999999, 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047812480000000001, 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1822 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325376+0000", 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325376+0000", 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325377+0000", 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2800000000000005e-07 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325382+0000", 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.643e-06 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325407+0000", 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4375e-05 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325413+0000", 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.326425+0000", 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001011781 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.326571+0000", 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014669499999999999 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.326980+0000", 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000408661 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328621+0000", 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016410489999999999 2024-09-16T11:13:25.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330045+0000", 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001424381 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330114+0000", 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8775000000000005e-05 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330157+0000", 2024-09-16T11:13:25.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099000000000002e-05 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1823 4.12 4:4fb601e4:::rbd_object_map.11b51c0df580:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.325422+0000", 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.0031504200000001, 2024-09-16T11:13:25.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0066789620000000001, 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1823 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325422+0000", 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325422+0000", 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325423+0000", 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325427+0000", 2024-09-16T11:13:25.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8759999999999998e-06 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325444+0000", 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7218999999999999e-05 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325460+0000", 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5888000000000001e-05 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.327547+0000", 2024-09-16T11:13:25.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020873279999999998 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.327648+0000", 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100312 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328064+0000", 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041622499999999999 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330030+0000", 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019663599999999999 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332001+0000", 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019708519999999999 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332062+0000", 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1063999999999997e-05 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332101+0000", 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8991e-05 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1824 4.12 4:4fb601e4:::rbd_object_map.11b51c0df580:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.0031078019999999, 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011589736999999999, 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.400 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1824 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999997e-07 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325468+0000", 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.694e-06 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325479+0000", 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1198e-05 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325483+0000", 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328787+0000", 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033047829999999999 2024-09-16T11:13:25.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328927+0000", 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013944200000000001 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329497+0000", 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057039799999999996 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.331046+0000", 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015491490000000001 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.336937+0000", 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058900660000000002 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337013+0000", 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6130999999999993e-05 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337054+0000", 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1789000000000003e-05 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:1825 4.1d e41/31 4:ba00c444:::rbd_data.11b51c0df580.0000000000000004:head v 41'21, mlcod=41'17)", 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.329414+0000", 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.99915885800000004, 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.007600491, 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.406 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329414+0000", 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329414+0000", 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329416+0000", 2024-09-16T11:13:25.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000001e-06 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329427+0000", 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329441+0000", 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4538999999999999e-05 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329455+0000", 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3655000000000001e-05 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329788+0000", 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033307399999999999 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329808+0000", 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0386000000000001e-05 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.336966+0000", 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00715761 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337014+0000", 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8140000000000003e-05 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.410 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2453 4.7 4:e7c9bd7a:::rbd_data.11b5220ac4a2.0000000000000019:head [truncate 1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.754255+0000", 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.57431756, 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038753070000000001, 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2453 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754255+0000", 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754255+0000", 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754257+0000", 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754267+0000", 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9070000000000002e-06 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754290+0000", 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3413000000000001e-05 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754300+0000", 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7040000000000006e-06 2024-09-16T11:13:25.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754398+0000", 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7714999999999995e-05 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754517+0000", 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000119392 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.754796+0000", 2024-09-16T11:13:25.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027852399999999999 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.756675+0000", 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001879443 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.757964+0000", 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012890110000000001 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.758043+0000", 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9103000000000005e-05 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.758130+0000", 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7060999999999999e-05 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2548 4.7 4:e7c9bd7a:::rbd_data.11b5220ac4a2.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.416 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.965716+0000", 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.36285643400000001, 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050843420000000004, 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2548 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965716+0000", 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965716+0000", 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965720+0000", 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965735+0000", 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5172e-05 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965787+0000", 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2711999999999999e-05 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965804+0000", 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7007000000000002e-05 2024-09-16T11:13:25.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965926+0000", 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121387 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966153+0000", 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022679199999999999 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966421+0000", 2024-09-16T11:13:25.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026855999999999999 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.968085+0000", 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00166344 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.970722+0000", 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002637089 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.970770+0000", 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8414000000000001e-05 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.970800+0000", 2024-09-16T11:13:25.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0269000000000001e-05 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2554 4.12 4:497c6bd0:::rbd_data.11b5220ac4a2.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.965889+0000", 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.36268324800000001, 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0051765250000000004, 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2554 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965889+0000", 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965889+0000", 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965891+0000", 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.756e-06 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965901+0000", 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5599999999999999e-06 2024-09-16T11:13:25.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965921+0000", 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0528999999999999e-05 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965931+0000", 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0356999999999999e-05 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966135+0000", 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000203492 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966307+0000", 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172449 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966562+0000", 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025463200000000002 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.968793+0000", 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022305620000000002 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.971016+0000", 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022230919999999999 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.971043+0000", 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7087e-05 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.971066+0000", 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3008999999999999e-05 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2656 4.7 4:e7f3171c:::rbd_data.11b5220ac4a2.000000000000002d:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:25.429 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.204276+0000", 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.124296799, 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015962905999999999, 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2656 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204276+0000", 2024-09-16T11:13:25.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204276+0000", 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204277+0000", 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.736e-06 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204295+0000", 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7858000000000001e-05 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204315+0000", 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860999999999998e-05 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204322+0000", 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0169999999999999e-06 2024-09-16T11:13:25.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204373+0000", 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0760999999999997e-05 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204478+0000", 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104802 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:25.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.207071+0000", 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002593026 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.219984+0000", 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012913508000000001 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220094+0000", 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010921699999999999 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220176+0000", 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2953000000000004e-05 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220239+0000", 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2167000000000003e-05 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.435 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:2656 4.7 e42/31 ondisk, result = 0)", 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.213033+0000", 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.115539737, 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072190750000000001, 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213033+0000", 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213033+0000", 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213035+0000", 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.841e-06 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213041+0000", 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4500000000000001e-06 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213061+0000", 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9947999999999999e-05 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213070+0000", 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4350000000000003e-06 2024-09-16T11:13:25.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220054+0000", 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069830400000000003 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220080+0000", 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6115999999999999e-05 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220252+0000", 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017224499999999999 2024-09-16T11:13:25.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.440 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.440 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.440 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:427 write 0x23d68c3 thru 0x23da164 (0x38a2 bytes) 2024-09-16T11:13:25.442 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:429 trunc from 0xc7d0297 to 0x7af35ad 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:431 read 0x46182cc thru 0x461a270 (0x1fa5 bytes) 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:432 trunc from 0x7af35ad to 0xe1232 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:436 read 0xbb223 thru 0xc34fc (0x82da bytes) 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:437 read 0x7fd7a thru 0x87c0f (0x7e96 bytes) 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:439 read 0x81ecd thru 0x90b9e (0xecd2 bytes) 2024-09-16T11:13:25.443 INFO:teuthology.orchestra.run.smithi162.stdout:440 punch from 0x67425 to 0x6927c, (0x1e57 bytes) 2024-09-16T11:13:25.444 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:25.437+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:25.455 INFO:teuthology.orchestra.run.smithi162.stdout:441 trunc from 0xe1232 to 0x6b57250 2024-09-16T11:13:25.497 INFO:teuthology.orchestra.run.smithi162.stdout:442 write 0xca0749b thru 0xca0f66f (0x81d5 bytes) 2024-09-16T11:13:25.504 INFO:teuthology.orchestra.run.smithi162.stdout:443 read 0x379aa50 thru 0x37a0a3e (0x5fef bytes) 2024-09-16T11:13:25.504 INFO:teuthology.orchestra.run.smithi162.stdout:446 write 0x6b0e627 thru 0x6b1c374 (0xdd4e bytes) 2024-09-16T11:13:25.512 INFO:teuthology.orchestra.run.smithi162.stdout:447 punch from 0xa139099 to 0xa13b518, (0x247f bytes) 2024-09-16T11:13:25.512 INFO:teuthology.orchestra.run.smithi162.stdout:449 read 0x52c2409 thru 0x52cd97a (0xb572 bytes) 2024-09-16T11:13:25.512 INFO:teuthology.orchestra.run.smithi162.stdout:451 write 0x72eb417 thru 0x72f9478 (0xe062 bytes) 2024-09-16T11:13:25.519 INFO:teuthology.orchestra.run.smithi162.stdout:452 read 0x4c1c50e thru 0x4c1f041 (0x2b34 bytes) 2024-09-16T11:13:25.520 INFO:teuthology.orchestra.run.smithi162.stdout:454 trunc from 0xca0f670 to 0xbed0fc2 2024-09-16T11:13:25.542 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:25.537+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:25.544 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.545 INFO:teuthology.orchestra.run.smithi162.stdout:457 trunc from 0xbed0fc2 to 0xbff5922 2024-09-16T11:13:25.559 INFO:teuthology.orchestra.run.smithi162.stdout:460 write 0x45b2abd thru 0x45bca03 (0x9f47 bytes) 2024-09-16T11:13:25.563 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:25.564 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:25.564 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:25.564 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:25.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:25.642 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:25.637+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:25.659 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:25.723 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:25.723 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:25.724 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:25.733 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:25.743 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:25.741+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:25.789 INFO:teuthology.orchestra.run.smithi162.stdout:461 writesame 0xda1ced1 thru 0xda238c5 data_size 0x7(0x69f5 bytes) 2024-09-16T11:13:25.789 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.789 INFO:teuthology.orchestra.run.smithi162.stdout:463 read 0xc44919f thru 0xc44e2d4 (0x5136 bytes) 2024-09-16T11:13:25.789 INFO:teuthology.orchestra.run.smithi162.stdout:464 read 0x30006cc thru 0x300d833 (0xd168 bytes) 2024-09-16T11:13:25.789 INFO:teuthology.orchestra.run.smithi162.stdout:465 writesame 0x3980573 thru 0x3983717 data_size 0x1(0x31a5 bytes) 2024-09-16T11:13:25.798 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:25.801 INFO:teuthology.orchestra.run.smithi162.stdout:466 read 0xa290b85 thru 0xa2922cb (0x1747 bytes) 2024-09-16T11:13:25.801 INFO:teuthology.orchestra.run.smithi162.stdout:467 writesame 0x49a31f5 thru 0x49ae133 data_size 0x77(0xaf3f bytes) 2024-09-16T11:13:25.808 INFO:teuthology.orchestra.run.smithi162.stdout:468 write 0x49d044f thru 0x49de1ea (0xdd9c bytes) 2024-09-16T11:13:25.814 INFO:teuthology.orchestra.run.smithi162.stdout:469 write 0xd47c68e thru 0xd48bea3 (0xf816 bytes) 2024-09-16T11:13:25.819 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:25.822 INFO:teuthology.orchestra.run.smithi162.stdout:470 write 0x45c317a thru 0x45d2d9e (0xfc25 bytes) 2024-09-16T11:13:25.828 INFO:teuthology.orchestra.run.smithi162.stdout:471 read 0x7771f4f thru 0x7776207 (0x42b9 bytes) 2024-09-16T11:13:25.829 INFO:teuthology.orchestra.run.smithi162.stdout:473 read 0x678d245 thru 0x679c9ac (0xf768 bytes) 2024-09-16T11:13:25.829 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:25.835 INFO:teuthology.orchestra.run.smithi162.stdout:475 trunc from 0xda238c6 to 0xa23a4bd 2024-09-16T11:13:25.835 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:25.835 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 31.004304604000001, 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:25.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:25.840 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.713729834, 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:25.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:25.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.705321546, 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:25.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:25.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:25.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:25.851 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:25.852 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:25.853 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:25.853 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:25.849+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:25.854 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:25.857 INFO:teuthology.orchestra.run.smithi162.stdout:476 read 0x5826523 thru 0x583313c (0xcc1a bytes) 2024-09-16T11:13:25.857 INFO:teuthology.orchestra.run.smithi162.stdout:477 write 0xef1cfb thru 0xf01090 (0xf396 bytes) 2024-09-16T11:13:25.864 INFO:teuthology.orchestra.run.smithi162.stdout:481 writesame 0x14219ab thru 0x1427a53 data_size 0x31(0x60a9 bytes) 2024-09-16T11:13:25.869 INFO:teuthology.orchestra.run.smithi162.stdout:484 write 0x819338 thru 0x824293 (0xaf5c bytes) 2024-09-16T11:13:25.874 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:25.882 INFO:teuthology.orchestra.run.smithi162.stdout:487 trunc from 0xa23a4bd to 0x3d2f877 2024-09-16T11:13:25.897 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:25.920 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:25.941 INFO:teuthology.orchestra.run.smithi162.stdout:488 writesame 0xe6c0853 thru 0xe6d01fa data_size 0xc(0xf9a8 bytes) 2024-09-16T11:13:25.941 INFO:teuthology.orchestra.run.smithi162.stdout:489 punch from 0xaac6eaf to 0xaacbce6, (0x4e37 bytes) 2024-09-16T11:13:25.942 INFO:teuthology.orchestra.run.smithi162.stdout:490 trunc from 0xe6d01fb to 0xcc21bed 2024-09-16T11:13:25.944 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:25.946 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:25.953 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:25.949+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:25.954 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:25.954 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:25.954 INFO:teuthology.orchestra.run.smithi162.stdout:491 writesame 0xad0b0fb thru 0xad13955 data_size 0x1(0x885b bytes) 2024-09-16T11:13:25.964 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:25.982 INFO:teuthology.orchestra.run.smithi162.stdout:493 writesame 0xbf2b736 thru 0xbf30492 data_size 0x11(0x4d5d bytes) 2024-09-16T11:13:25.989 INFO:teuthology.orchestra.run.smithi162.stdout:495 write 0x42e6845 thru 0x42f23a0 (0xbb5c bytes) 2024-09-16T11:13:25.996 INFO:teuthology.orchestra.run.smithi162.stdout:497 write 0x901dd41 thru 0x90273a0 (0x9660 bytes) 2024-09-16T11:13:26.012 INFO:teuthology.orchestra.run.smithi162.stdout:498 trunc from 0xcc21bed to 0x75ae8d2 2024-09-16T11:13:26.035 INFO:teuthology.orchestra.run.smithi162.stdout:499 read 0x6030b77 thru 0x6039315 (0x879f bytes) 2024-09-16T11:13:26.053 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:26.049+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:26.056 INFO:teuthology.orchestra.run.smithi162.stdout:500 write 0xc6cd4e1 thru 0xc6dc2a0 (0xedc0 bytes) 2024-09-16T11:13:26.058 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.058 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.063 INFO:teuthology.orchestra.run.smithi162.stdout:501 write 0x687b665 thru 0x687c7b2 (0x114e bytes) 2024-09-16T11:13:26.068 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:26.084 INFO:teuthology.orchestra.run.smithi162.stdout:502 write 0xd452c7f thru 0xd462a71 (0xfdf3 bytes) 2024-09-16T11:13:26.088 INFO:teuthology.orchestra.run.smithi162.stdout:503 punch from 0x4d83664 to 0x4d8cec8, (0x9864 bytes) 2024-09-16T11:13:26.088 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:26.088 INFO:teuthology.orchestra.run.smithi162.stdout:505 write 0x31e9b86 thru 0x31f2804 (0x8c7f bytes) 2024-09-16T11:13:26.095 INFO:teuthology.orchestra.run.smithi162.stdout:509 write 0x2b7d337 thru 0x2b81d46 (0x4a10 bytes) 2024-09-16T11:13:26.112 INFO:teuthology.orchestra.run.smithi162.stdout:510 trunc from 0xd462a72 to 0x9315dec 2024-09-16T11:13:26.135 INFO:teuthology.orchestra.run.smithi162.stdout:511 write 0x150efc5 thru 0x1512c09 (0x3c45 bytes) 2024-09-16T11:13:26.145 INFO:teuthology.orchestra.run.smithi162.stdout:512 trunc from 0x9315dec to 0x72f9a1d 2024-09-16T11:13:26.154 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:26.149+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:26.159 INFO:teuthology.orchestra.run.smithi162.stdout:513 write 0xaa4282 thru 0xab08db (0xc65a bytes) 2024-09-16T11:13:26.162 INFO:teuthology.orchestra.run.smithi162.stdout:514 write 0x5c12d1b thru 0x5c1341c (0x702 bytes) 2024-09-16T11:13:26.170 INFO:teuthology.orchestra.run.smithi162.stdout:515 write 0x5cd632d thru 0x5ce3334 (0xd008 bytes) 2024-09-16T11:13:26.179 INFO:teuthology.orchestra.run.smithi162.stdout:516 punch from 0x103188c to 0x10347d4, (0x2f48 bytes) 2024-09-16T11:13:26.179 INFO:teuthology.orchestra.run.smithi162.stdout:517 trunc from 0x72f9a1d to 0x79cd481 2024-09-16T11:13:26.190 INFO:teuthology.orchestra.run.smithi162.stdout:518 read 0x4d1332c thru 0x4d1b8e9 (0x85be bytes) 2024-09-16T11:13:26.192 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.192 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.200 INFO:teuthology.orchestra.run.smithi162.stdout:520 trunc from 0x79cd481 to 0x1b6a464 2024-09-16T11:13:26.206 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:26.229 INFO:teuthology.orchestra.run.smithi162.stdout:521 punch from 0x396402 to 0x39fa36, (0x9634 bytes) 2024-09-16T11:13:26.229 INFO:teuthology.orchestra.run.smithi162.stdout:523 read 0x27bbb thru 0x281a9 (0x5ef bytes) 2024-09-16T11:13:26.246 INFO:teuthology.orchestra.run.smithi162.stdout:524 write 0x416b09e thru 0x4171036 (0x5f99 bytes) 2024-09-16T11:13:26.255 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:26.249+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:26.272 INFO:teuthology.orchestra.run.smithi162.stdout:525 write 0x59fbfe8 thru 0x5a0ad4e (0xed67 bytes) 2024-09-16T11:13:26.283 INFO:teuthology.orchestra.run.smithi162.stdout:529 punch from 0x3117df2 to 0x31185dc, (0x7ea bytes) 2024-09-16T11:13:26.312 INFO:teuthology.orchestra.run.smithi162.stdout:530 write 0xcb39216 thru 0xcb3eca5 (0x5a90 bytes) 2024-09-16T11:13:26.317 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.317 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.318 INFO:teuthology.orchestra.run.smithi162.stdout:531 read 0x2c2422e thru 0x2c306da (0xc4ad bytes) 2024-09-16T11:13:26.319 INFO:teuthology.orchestra.run.smithi162.stdout:532 read 0x967208e thru 0x9681a45 (0xf9b8 bytes) 2024-09-16T11:13:26.319 INFO:teuthology.orchestra.run.smithi162.stdout:533 punch from 0xb73cdef to 0xb74897c, (0xbb8d bytes) 2024-09-16T11:13:26.326 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:26.340 INFO:teuthology.orchestra.run.smithi162.stdout:537 trunc from 0xcb3eca6 to 0xbd46c3 2024-09-16T11:13:26.356 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:26.349+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:26.365 INFO:teuthology.orchestra.run.smithi162.stdout:538 write 0x6bb8ce thru 0x6c5282 (0x99b5 bytes) 2024-09-16T11:13:26.375 INFO:teuthology.orchestra.run.smithi162.stdout:539 read 0x7471d9 thru 0x74986d (0x2695 bytes) 2024-09-16T11:13:26.376 INFO:teuthology.orchestra.run.smithi162.stdout:541 read 0x7bf7b thru 0x8a9aa (0xea30 bytes) 2024-09-16T11:13:26.377 INFO:teuthology.orchestra.run.smithi162.stdout:542 punch from 0x836ec2 to 0x839c2c, (0x2d6a bytes) 2024-09-16T11:13:26.407 INFO:teuthology.orchestra.run.smithi162.stdout:544 write 0x814fa69 thru 0x815c432 (0xc9ca bytes) 2024-09-16T11:13:26.421 INFO:teuthology.orchestra.run.smithi162.stdout:545 trunc from 0x815c433 to 0xb183a90 2024-09-16T11:13:26.426 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.426 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.435 INFO:teuthology.orchestra.run.smithi162.stdout:547 writesame 0x7010dd7 thru 0x7013701 data_size 0x3(0x292b bytes) 2024-09-16T11:13:26.436 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:26.442 INFO:teuthology.orchestra.run.smithi162.stdout:551 write 0x2fb0969 thru 0x2fb1e77 (0x150f bytes) 2024-09-16T11:13:26.448 INFO:teuthology.orchestra.run.smithi162.stdout:552 read 0xa0bf477 thru 0xa0c1e0f (0x2999 bytes) 2024-09-16T11:13:26.448 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:26.448 INFO:teuthology.orchestra.run.smithi162.stdout:554 write 0x7176425 thru 0x71820ff (0xbcdb bytes) 2024-09-16T11:13:26.456 INFO:teuthology.orchestra.run.smithi162.stdout:555 read 0x6b6ed4a thru 0x6b7074e (0x1a05 bytes) 2024-09-16T11:13:26.456 INFO:teuthology.orchestra.run.smithi162.stdout:556 read 0xf6d552 thru 0xf7ca33 (0xf4e2 bytes) 2024-09-16T11:13:26.456 INFO:teuthology.orchestra.run.smithi162.stdout:557 punch from 0x1b1bba2 to 0x1b202c9, (0x4727 bytes) 2024-09-16T11:13:26.457 INFO:teuthology.orchestra.run.smithi162.stdout:560 write 0x2b09140 thru 0x2b098c4 (0x785 bytes) 2024-09-16T11:13:26.457 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:26.453+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:26.458 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:26.458 INFO:teuthology.orchestra.run.smithi162.stdout:563 punch from 0x53b340 to 0x549f5b, (0xec1b bytes) 2024-09-16T11:13:26.481 INFO:teuthology.orchestra.run.smithi162.stdout:564 trunc from 0xb183a90 to 0x612479 2024-09-16T11:13:26.519 INFO:teuthology.orchestra.run.smithi162.stdout:565 trunc from 0x612479 to 0x6690087 2024-09-16T11:13:26.529 INFO:teuthology.orchestra.run.smithi162.stdout:566 punch from 0x126b7b2 to 0x127b4c2, (0xfd10 bytes) 2024-09-16T11:13:26.529 INFO:teuthology.orchestra.run.smithi162.stdout:567 punch from 0x30ecce3 to 0x30f2828, (0x5b45 bytes) 2024-09-16T11:13:26.529 INFO:teuthology.orchestra.run.smithi162.stdout:568 write 0x293391d thru 0x2937e6a (0x454e bytes) 2024-09-16T11:13:26.532 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.533 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.542 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:26.560 INFO:teuthology.orchestra.run.smithi162.stdout:569 write 0xb8b4c8e thru 0xb8c1786 (0xcaf9 bytes) 2024-09-16T11:13:26.568 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:26.568 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:26.568 INFO:teuthology.orchestra.run.smithi162.stdout:573 write 0xa9bae19 thru 0xa9c7ac8 (0xccb0 bytes) 2024-09-16T11:13:26.576 INFO:teuthology.orchestra.run.smithi162.stdout:574 writesame 0x3650437 thru 0x36552f7 data_size 0x1(0x4ec1 bytes) 2024-09-16T11:13:26.587 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:26.585+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:26.597 INFO:teuthology.orchestra.run.smithi162.stdout:575 read 0x6bc5402 thru 0x6bc5d54 (0x953 bytes) 2024-09-16T11:13:26.597 INFO:teuthology.orchestra.run.smithi162.stdout:576 write 0xb6ac6d7 thru 0xb6b3c10 (0x753a bytes) 2024-09-16T11:13:26.600 INFO:teuthology.orchestra.run.smithi162.stdout:577 writesame 0x87e6dd0 thru 0x87eb35d data_size 0x1d(0x458e bytes) 2024-09-16T11:13:26.606 INFO:teuthology.orchestra.run.smithi162.stdout:579 read 0x60f3d3e thru 0x60fdf11 (0xa1d4 bytes) 2024-09-16T11:13:26.606 INFO:teuthology.orchestra.run.smithi162.stdout:582 read 0x913e21b thru 0x914304a (0x4e30 bytes) 2024-09-16T11:13:26.607 INFO:teuthology.orchestra.run.smithi162.stdout:583 write 0x5a8308d thru 0x5a85dc8 (0x2d3c bytes) 2024-09-16T11:13:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:584 trunc from 0xb8c1787 to 0xbe4d76b 2024-09-16T11:13:26.626 INFO:teuthology.orchestra.run.smithi162.stdout:585 write 0x997fbe9 thru 0x998d467 (0xd87f bytes) 2024-09-16T11:13:26.634 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:26.634 INFO:teuthology.orchestra.run.smithi162.stdout:587 clone 5 order 21 su 131072 sc 3 2024-09-16T11:13:26.645 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.646 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.655 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:26.658 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:26.653+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:26.751 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.751 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.759 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:26.757+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:26.760 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:26.858 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.858 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.859 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:26.857+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:26.867 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:26.960 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:26.957+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:26.966 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:26.967 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:26.976 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:27.061 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:27.057+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:27.076 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.076 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.085 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:27.163 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:27.157+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:27.200 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:13:27.240 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.240 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.250 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:27.264 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:27.261+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:27.351 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.352 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.360 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:27.365 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:27.361+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:27.466 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:27.461+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:27.507 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T11:13:27.509 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.509 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.518 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:27.567 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:27.561+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:27.586 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.587 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.587 INFO:teuthology.orchestra.run.smithi162.stdout:590 read 0x5cca9a6 thru 0x5cd9221 (0xe87c bytes) 2024-09-16T11:13:27.596 INFO:teuthology.orchestra.run.smithi162.stdout:591 writesame 0x1e27a63 thru 0x1e2ee00 data_size 0x3(0x739e bytes) 2024-09-16T11:13:27.607 INFO:teuthology.orchestra.run.smithi162.stdout:592 read 0x80d2fee thru 0x80dc2fa (0x930d bytes) 2024-09-16T11:13:27.618 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.618 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.622 INFO:teuthology.orchestra.run.smithi162.stdout:593 writesame 0xc25dc09 thru 0xc25eab1 data_size 0x9(0xea9 bytes) 2024-09-16T11:13:27.627 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:27.638 INFO:teuthology.orchestra.run.smithi162.stdout:594 trunc from 0xc25eab2 to 0x5a95382 2024-09-16T11:13:27.667 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:27.665+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:27.669 INFO:teuthology.orchestra.run.smithi162.stdout:595 write 0x2e50448 thru 0x2e52b52 (0x270b bytes) 2024-09-16T11:13:27.677 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.677 INFO:teuthology.orchestra.run.smithi162.stdout:598 write 0x50d31b0 thru 0x50e0c67 (0xdab8 bytes) 2024-09-16T11:13:27.686 INFO:teuthology.orchestra.run.smithi162.stdout:599 write 0x5743c57 thru 0x5747983 (0x3d2d bytes) 2024-09-16T11:13:27.694 INFO:teuthology.orchestra.run.smithi162.stdout:600 punch from 0x449ac86 to 0x44a8bf5, (0xdf6f bytes) 2024-09-16T11:13:27.694 INFO:teuthology.orchestra.run.smithi162.stdout:601 read 0x4d6b329 thru 0x4d7999c (0xe674 bytes) 2024-09-16T11:13:27.694 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.694 INFO:teuthology.orchestra.run.smithi162.stdout:603 read 0x52d8177 thru 0x52da07a (0x1f04 bytes) 2024-09-16T11:13:27.695 INFO:teuthology.orchestra.run.smithi162.stdout:604 punch from 0x19b179c to 0x19b9e03, (0x8667 bytes) 2024-09-16T11:13:27.695 INFO:teuthology.orchestra.run.smithi162.stdout:611 write 0x3f24e27 thru 0x3f28deb (0x3fc5 bytes) 2024-09-16T11:13:27.715 INFO:teuthology.orchestra.run.smithi162.stdout:612 write 0x6e2067a thru 0x6e2c986 (0xc30d bytes) 2024-09-16T11:13:27.722 INFO:teuthology.orchestra.run.smithi162.stdout:613 read 0x65f857b thru 0x65fd939 (0x53bf bytes) 2024-09-16T11:13:27.722 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.722 INFO:teuthology.orchestra.run.smithi162.stdout:616 write 0x5d4296b thru 0x5d4f053 (0xc6e9 bytes) 2024-09-16T11:13:27.724 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.725 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.725 INFO:teuthology.orchestra.run.smithi162.stdout:619 punch from 0x567eb19 to 0x56854dc, (0x69c3 bytes) 2024-09-16T11:13:27.728 INFO:teuthology.orchestra.run.smithi162.stdout:621 trunc from 0x6e2c987 to 0x4d996e4 2024-09-16T11:13:27.740 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.740 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.750 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:27.756 INFO:teuthology.orchestra.run.smithi162.stdout:622 write 0x635d53 thru 0x63d63c (0x78ea bytes) 2024-09-16T11:13:27.769 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:27.765+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:27.777 INFO:teuthology.orchestra.run.smithi162.stdout:624 writesame 0x648be54 thru 0x6491c58 data_size 0x3(0x5e05 bytes) 2024-09-16T11:13:27.777 INFO:teuthology.orchestra.run.smithi162.stdout:625 read 0xa83b4b thru 0xa8922f (0x56e5 bytes) 2024-09-16T11:13:27.802 INFO:teuthology.orchestra.run.smithi162.stdout:626 writesame 0xe3759f0 thru 0xe379f07 data_size 0x4(0x4518 bytes) 2024-09-16T11:13:27.802 INFO:teuthology.orchestra.run.smithi162.stdout:627 writesame 0x682e162 thru 0x6833894 data_size 0x7(0x5733 bytes) 2024-09-16T11:13:27.812 INFO:teuthology.orchestra.run.smithi162.stdout:628 read 0x7fc8d7e thru 0x7fd4573 (0xb7f6 bytes) 2024-09-16T11:13:27.812 INFO:teuthology.orchestra.run.smithi162.stdout:630 write 0x9624f18 thru 0x962ace7 (0x5dd0 bytes) 2024-09-16T11:13:27.818 INFO:teuthology.orchestra.run.smithi162.stdout:633 write 0x9f8c158 thru 0x9f9b991 (0xf83a bytes) 2024-09-16T11:13:27.825 INFO:teuthology.orchestra.run.smithi162.stdout:637 punch from 0x2ebe6ba to 0x2ec0cfd, (0x2643 bytes) 2024-09-16T11:13:27.825 INFO:teuthology.orchestra.run.smithi162.stdout:638 read 0x40dc0bd thru 0x40df76c (0x36b0 bytes) 2024-09-16T11:13:27.825 INFO:teuthology.orchestra.run.smithi162.stdout:640 punch from 0x854cf79 to 0x8554ecb, (0x7f52 bytes) 2024-09-16T11:13:27.825 INFO:teuthology.orchestra.run.smithi162.stdout:641 read 0x77876de thru 0x7792642 (0xaf65 bytes) 2024-09-16T11:13:27.841 INFO:teuthology.orchestra.run.smithi162.stdout:642 write 0xe5d79d2 thru 0xe5e1bd8 (0xa207 bytes) 2024-09-16T11:13:27.851 INFO:teuthology.orchestra.run.smithi162.stdout:644 read 0x1e8b691 thru 0x1e9b5da (0xff4a bytes) 2024-09-16T11:13:27.859 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:27.859 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:27.860 INFO:teuthology.orchestra.run.smithi162.stdout:645 trunc from 0xe5e1bd9 to 0x998ccde 2024-09-16T11:13:27.870 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:27.870 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:27.865+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:27.897 INFO:teuthology.orchestra.run.smithi162.stdout:648 write 0xcd9606e thru 0xcd9a423 (0x43b6 bytes) 2024-09-16T11:13:27.903 INFO:teuthology.orchestra.run.smithi162.stdout:651 read 0x1a48a20 thru 0x1a50b53 (0x8134 bytes) 2024-09-16T11:13:27.922 INFO:teuthology.orchestra.run.smithi162.stdout:652 trunc from 0xcd9a424 to 0x28dcbd7 2024-09-16T11:13:27.952 INFO:teuthology.orchestra.run.smithi162.stdout:653 read 0x9adce5 thru 0x9b009c (0x23b8 bytes) 2024-09-16T11:13:27.958 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:27.981 INFO:teuthology.orchestra.run.smithi162.stdout:654 write 0xe5af870 thru 0xe5b537f (0x5b10 bytes) 2024-09-16T11:13:27.981 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:27.977+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:27.988 INFO:teuthology.orchestra.run.smithi162.stdout:655 read 0x1e32f9a thru 0x1e3e525 (0xb58c bytes) 2024-09-16T11:13:27.989 INFO:teuthology.orchestra.run.smithi162.stdout:656 read 0xa75faac thru 0xa76f21e (0xf773 bytes) 2024-09-16T11:13:27.989 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:27.989 INFO:teuthology.orchestra.run.smithi162.stdout:660 read 0x9410049 thru 0x941b5b9 (0xb571 bytes) 2024-09-16T11:13:28.005 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:28.005 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:28.009 INFO:teuthology.orchestra.run.smithi162.stdout:661 trunc from 0xe5b5380 to 0x1c59d46 2024-09-16T11:13:28.017 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:28.035 INFO:teuthology.orchestra.run.smithi162.stdout:662 read 0x215a71 thru 0x2177e2 (0x1d72 bytes) 2024-09-16T11:13:28.065 INFO:teuthology.orchestra.run.smithi162.stdout:664 write 0xe6d5633 thru 0xe6db883 (0x6251 bytes) 2024-09-16T11:13:28.071 INFO:teuthology.orchestra.run.smithi162.stdout:665 write 0xddefe6f thru 0xddf2af6 (0x2c88 bytes) 2024-09-16T11:13:28.071 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:28.065+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:28.078 INFO:teuthology.orchestra.run.smithi162.stdout:666 writesame 0x8761f18 thru 0x876a038 data_size 0x3(0x8121 bytes) 2024-09-16T11:13:28.090 INFO:teuthology.orchestra.run.smithi162.stdout:667 punch from 0x20c742c to 0x20d11a8, (0x9d7c bytes) 2024-09-16T11:13:28.090 INFO:teuthology.orchestra.run.smithi162.stdout:668 write 0x18909c0 thru 0x189a293 (0x98d4 bytes) 2024-09-16T11:13:28.094 INFO:teuthology.orchestra.run.smithi162.stdout:669 read 0x7eecd24 thru 0x7ef66ac (0x9989 bytes) 2024-09-16T11:13:28.094 INFO:teuthology.orchestra.run.smithi162.stdout:670 writesame 0x51b410b thru 0x51bad0b data_size 0x1(0x6c01 bytes) 2024-09-16T11:13:28.136 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:28.136 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:28.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:28.150 INFO:teuthology.orchestra.run.smithi162.stdout:671 trunc from 0xe6db884 to 0xf254b1 2024-09-16T11:13:28.184 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.184 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:28.177+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:28.209 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:13:28.212 INFO:teuthology.orchestra.run.smithi162.stdout:673 write 0xa65ee76 thru 0xa66e456 (0xf5e1 bytes) 2024-09-16T11:13:28.233 INFO:teuthology.orchestra.run.smithi162.stdout:675 write 0xd847d5a thru 0xd85537f (0xd626 bytes) 2024-09-16T11:13:28.239 INFO:teuthology.orchestra.run.smithi162.stdout:676 read 0xaed273a thru 0xaed3c44 (0x150b bytes) 2024-09-16T11:13:28.239 INFO:teuthology.orchestra.run.smithi162.stdout:677 read 0x7044a77 thru 0x704628e (0x1818 bytes) 2024-09-16T11:13:28.239 INFO:teuthology.orchestra.run.smithi162.stdout:678 writesame 0x7075a35 thru 0x7080a7b data_size 0x1(0xb047 bytes) 2024-09-16T11:13:28.264 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:28.265 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:28.273 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:28.269+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:28.274 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:28.277 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.277 INFO:teuthology.orchestra.run.smithi162.stdout:680 punch from 0xcf1260a to 0xcf128d9, (0x2cf bytes) 2024-09-16T11:13:28.278 INFO:teuthology.orchestra.run.smithi162.stdout:682 writesame 0x74cc828 thru 0x74dbff6 data_size 0x1(0xf7cf bytes) 2024-09-16T11:13:28.317 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:28.327 INFO:teuthology.orchestra.run.smithi162.stdout:683 writesame 0xc6de2f5 thru 0xc6eda97 data_size 0x1f(0xf7a3 bytes) 2024-09-16T11:13:28.336 INFO:teuthology.orchestra.run.smithi162.stdout:684 write 0x840dc3c thru 0x8419d01 (0xc0c6 bytes) 2024-09-16T11:13:28.341 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:28.343 INFO:teuthology.orchestra.run.smithi162.stdout:685 read 0x9ddca6f thru 0x9de19f0 (0x4f82 bytes) 2024-09-16T11:13:28.344 INFO:teuthology.orchestra.run.smithi162.stdout:687 writesame 0x10af8cf thru 0x10bea4c data_size 0x1(0xf17e bytes) 2024-09-16T11:13:28.361 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:28.374 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:28.369+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:28.383 INFO:teuthology.orchestra.run.smithi162.stdout:689 punch from 0x57924bd to 0x5799356, (0x6e99 bytes) 2024-09-16T11:13:28.383 INFO:teuthology.orchestra.run.smithi162.stdout:690 writesame 0x35df4a4 thru 0x35ebbf5 data_size 0x1f(0xc752 bytes) 2024-09-16T11:13:28.392 INFO:teuthology.orchestra.run.smithi162.stdout:691 read 0x543416b thru 0x5436f8b (0x2e21 bytes) 2024-09-16T11:13:28.393 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:28.393 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:28.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:28.393 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:544 4.a e38/31 4:57f43eeb:::rbd_object_map.11b52605a025:head v 38'14, mlcod=38'12)", 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.300645+0000", 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.0901096159999994, 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042862730000000002, 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300645+0000", 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300645+0000", 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300647+0000", 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.547e-06 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300688+0000", 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1341999999999998e-05 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300775+0000", 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7471000000000003e-05 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.300827+0000", 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1592999999999999e-05 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.301179+0000", 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035227199999999999 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.301251+0000", 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2025000000000002e-05 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.304804+0000", 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035531460000000001 2024-09-16T11:13:28.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.304931+0000", 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012687700000000001 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:756 4.b e39/31 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head v 39'5, mlcod=38'2)", 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.598371287, 2024-09-16T11:13:28.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027783187000000001, 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792386+0000", 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.667e-06 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799237+0000", 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006850737 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799272+0000", 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5147000000000001e-05 2024-09-16T11:13:28.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799284+0000", 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2391999999999999e-05 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799389+0000", 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104759 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799444+0000", 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5118000000000003e-05 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.819973+0000", 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020528497 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820167+0000", 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019387000000000001 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:828 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13441105~9082] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:28.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.5335555420000002, 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042301801999999999, 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 828 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857203+0000", 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299999999999999e-06 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862148+0000", 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049449400000000001 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862190+0000", 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2153000000000001e-05 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862203+0000", 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2604e-05 2024-09-16T11:13:28.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862330+0000", 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012740900000000001 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862670+0000", 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033967399999999999 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.863645+0000", 2024-09-16T11:13:28.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00097586600000000004 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.884164+0000", 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020518723999999999 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899338+0000", 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015173623000000001 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899435+0000", 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7671999999999996e-05 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899501+0000", 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5407000000000006e-05 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:871 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [stat,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:28.409 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.918369+0000", 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.4723852480000001, 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096987259999999995, 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 871 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918369+0000", 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918369+0000", 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918373+0000", 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918392+0000", 2024-09-16T11:13:28.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9326000000000001e-05 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918427+0000", 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4938999999999999e-05 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918444+0000", 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6534000000000001e-05 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918601+0000", 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000157186 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918929+0000", 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032847799999999999 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919317+0000", 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038708900000000002 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924671+0000", 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053548659999999998 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.927967+0000", 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032950940000000001 2024-09-16T11:13:28.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.928029+0000", 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2204000000000003e-05 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.928068+0000", 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9359e-05 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:874 4.f 4:f1087b7b:::rbd_data.11b585795d6b.0000000000000005:head [stat,zero 8388608~8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.918502+0000", 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.4722524740000003, 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004962574, 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 874 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.416 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918502+0000", 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918502+0000", 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918504+0000", 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3519999999999999e-06 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918510+0000", 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6989999999999997e-06 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918534+0000", 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3957000000000001e-05 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918545+0000", 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0671e-05 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.918740+0000", 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000195236 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919007+0000", 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026729400000000001 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.923465+0000", 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044573649999999996 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:891 4.9 e39/31 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head v 39'1, mlcod=0'0)", 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.935936+0000", 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.4548187539999997, 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0097245609999999996, 2024-09-16T11:13:28.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.935936+0000", 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.935936+0000", 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.935938+0000", 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.933e-06 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939444+0000", 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035060769999999998 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939460+0000", 2024-09-16T11:13:28.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.605e-05 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939467+0000", 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.046e-06 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939500+0000", 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3151999999999997e-05 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.939529+0000", 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9354999999999999e-05 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.945603+0000", 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060735329999999999 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.945660+0000", 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7414999999999998e-05 2024-09-16T11:13:28.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:921 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.991351+0000", 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.3994034099999997, 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065742999999999999, 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.425 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 921 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991351+0000", 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991351+0000", 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991353+0000", 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1069999999999999e-06 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991364+0000", 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0560000000000001e-05 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991398+0000", 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4001999999999997e-05 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991410+0000", 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.217e-05 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991530+0000", 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120072 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991676+0000", 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000145725 2024-09-16T11:13:28.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.991841+0000", 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016474799999999999 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.996861+0000", 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050203219999999998 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.997546+0000", 2024-09-16T11:13:28.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068512200000000003 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.997811+0000", 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026539000000000001 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.997926+0000", 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114082 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:923 4.b e39/31 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head v 39'8, mlcod=39'6)", 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.995274+0000", 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 7.3954807840000001, 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045649009999999997, 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995274+0000", 2024-09-16T11:13:28.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995274+0000", 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995275+0000", 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.282e-06 2024-09-16T11:13:28.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995285+0000", 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7340000000000003e-06 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995315+0000", 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9794e-05 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995325+0000", 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0509000000000001e-05 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995464+0000", 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013907899999999999 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.995504+0000", 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9501e-05 2024-09-16T11:13:28.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999779+0000", 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004275048 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999839+0000", 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9954000000000002e-05 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1816 4.f 4:f1d27fab:::rbd_data.11b51c0df580.000000000000000a:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.312598+0000", 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0781566309999997, 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0077270070000000001, 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1816 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.436 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312598+0000", 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312598+0000", 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312600+0000", 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2749999999999998e-06 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312610+0000", 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0039e-05 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312634+0000", 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3274000000000001e-05 2024-09-16T11:13:28.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312648+0000", 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4163e-05 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312714+0000", 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5973e-05 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312793+0000", 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9536999999999994e-05 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312980+0000", 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000186319 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.314393+0000", 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001413032 2024-09-16T11:13:28.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320230+0000", 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058371040000000001 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320276+0000", 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6631999999999997e-05 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320325+0000", 2024-09-16T11:13:28.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8659000000000003e-05 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:1819 4.1d e41/31 4:ba69c8d3:::rbd_data.11b51c0df580.000000000000000f:head v 41'19, mlcod=40'14)", 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.315567+0000", 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0751876359999999, 2024-09-16T11:13:28.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0041240449999999998, 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315567+0000", 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315567+0000", 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315568+0000", 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6000000000000002e-07 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315575+0000", 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3359999999999997e-06 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315590+0000", 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4979e-05 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315597+0000", 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4500000000000001e-06 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315639+0000", 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2361000000000003e-05 2024-09-16T11:13:28.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315657+0000", 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7691999999999999e-05 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319601+0000", 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039445579999999999 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319691+0000", 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9808999999999993e-05 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1822 4.12 4:4fb601e4:::rbd_object_map.11b51c0df580:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.325376+0000", 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0653787079999999, 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047812480000000001, 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1822 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325376+0000", 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325376+0000", 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325377+0000", 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2800000000000005e-07 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325382+0000", 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.643e-06 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325407+0000", 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4375e-05 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325413+0000", 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.326425+0000", 2024-09-16T11:13:28.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001011781 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.326571+0000", 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014669499999999999 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.326980+0000", 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000408661 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328621+0000", 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016410489999999999 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330045+0000", 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001424381 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330114+0000", 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8775000000000005e-05 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330157+0000", 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099000000000002e-05 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1823 4.12 4:4fb601e4:::rbd_object_map.11b51c0df580:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.325422+0000", 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0653325059999998, 2024-09-16T11:13:28.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0066789620000000001, 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1823 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325422+0000", 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325422+0000", 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325423+0000", 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.47e-07 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325427+0000", 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8759999999999998e-06 2024-09-16T11:13:28.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325444+0000", 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7218999999999999e-05 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325460+0000", 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5888000000000001e-05 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.327547+0000", 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020873279999999998 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.327648+0000", 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100312 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328064+0000", 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00041622499999999999 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.330030+0000", 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019663599999999999 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332001+0000", 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019708519999999999 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332062+0000", 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1063999999999997e-05 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332101+0000", 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8991e-05 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1824 4.12 4:4fb601e4:::rbd_object_map.11b51c0df580:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0652898879999997, 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011589736999999999, 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.459 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1824 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325465+0000", 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999997e-07 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325468+0000", 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.694e-06 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325479+0000", 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1198e-05 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.325483+0000", 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328787+0000", 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033047829999999999 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328927+0000", 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013944200000000001 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329497+0000", 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057039799999999996 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.331046+0000", 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015491490000000001 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.336937+0000", 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058900660000000002 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337013+0000", 2024-09-16T11:13:28.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6130999999999993e-05 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337054+0000", 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1789000000000003e-05 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:1825 4.1d e41/31 4:ba00c444:::rbd_data.11b51c0df580.0000000000000004:head v 41'21, mlcod=41'17)", 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.329414+0000", 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0613409440000003, 2024-09-16T11:13:28.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.007600491, 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329414+0000", 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329414+0000", 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329416+0000", 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000001e-06 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329427+0000", 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0526999999999999e-05 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329441+0000", 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4538999999999999e-05 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329455+0000", 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3655000000000001e-05 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329788+0000", 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033307399999999999 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329808+0000", 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0386000000000001e-05 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.336966+0000", 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00715761 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337014+0000", 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8140000000000003e-05 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.469 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2548 4.7 4:e7c9bd7a:::rbd_data.11b5220ac4a2.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.965716+0000", 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.4250385200000002, 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050843420000000004, 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2548 2024-09-16T11:13:28.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965716+0000", 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965716+0000", 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965720+0000", 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965735+0000", 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5172e-05 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965787+0000", 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2711999999999999e-05 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965804+0000", 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7007000000000002e-05 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965926+0000", 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121387 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966153+0000", 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022679199999999999 2024-09-16T11:13:28.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966421+0000", 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026855999999999999 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.968085+0000", 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00166344 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.970722+0000", 2024-09-16T11:13:28.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002637089 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.970770+0000", 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8414000000000001e-05 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.970800+0000", 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0269000000000001e-05 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2554 4.12 4:497c6bd0:::rbd_data.11b5220ac4a2.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.965889+0000", 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.4248653340000001, 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0051765250000000004, 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2554 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965889+0000", 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965889+0000", 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965891+0000", 2024-09-16T11:13:28.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.756e-06 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965901+0000", 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5599999999999999e-06 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965921+0000", 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0528999999999999e-05 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.965931+0000", 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0356999999999999e-05 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966135+0000", 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000203492 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966307+0000", 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172449 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.966562+0000", 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025463200000000002 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.968793+0000", 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022305620000000002 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.971016+0000", 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022230919999999999 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.971043+0000", 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7087e-05 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.971066+0000", 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3008999999999999e-05 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2656 4.7 4:e7f3171c:::rbd_data.11b5220ac4a2.000000000000002d:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.204276+0000", 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1864788850000001, 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015962905999999999, 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2656 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.482 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204276+0000", 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204276+0000", 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204277+0000", 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.736e-06 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204295+0000", 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7858000000000001e-05 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204315+0000", 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860999999999998e-05 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204322+0000", 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0169999999999999e-06 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204373+0000", 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0760999999999997e-05 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.204478+0000", 2024-09-16T11:13:28.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104802 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.207071+0000", 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002593026 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.219984+0000", 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012913508000000001 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220094+0000", 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010921699999999999 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220176+0000", 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2953000000000004e-05 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220239+0000", 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2167000000000003e-05 2024-09-16T11:13:28.487 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:2656 4.7 e42/31 ondisk, result = 0)", 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.213033+0000", 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1777218230000002, 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0072190750000000001, 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213033+0000", 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213033+0000", 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213035+0000", 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.841e-06 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213041+0000", 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4500000000000001e-06 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213061+0000", 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9947999999999999e-05 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.213070+0000", 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4350000000000003e-06 2024-09-16T11:13:28.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220054+0000", 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069830400000000003 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220080+0000", 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6115999999999999e-05 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.220252+0000", 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017224499999999999 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2722 4.7 4:e7f3171c:::rbd_data.11b5220ac4a2.000000000000002d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.389842+0000", 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.0009121830000001, 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005685509, 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2722 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389842+0000", 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389842+0000", 2024-09-16T11:13:28.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389843+0000", 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9000000000000005e-07 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389849+0000", 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389865+0000", 2024-09-16T11:13:28.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389875+0000", 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.04e-05 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.389987+0000", 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011158999999999999 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.390159+0000", 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017172199999999999 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.390392+0000", 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023356599999999999 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.391986+0000", 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00159381 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.395410+0000", 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034243860000000002 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.395477+0000", 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6662000000000007e-05 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.395528+0000", 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0936000000000002e-05 2024-09-16T11:13:28.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2810 4.4 4:24a0fcb5:::rbd_data.11b5220ac4a2.0000000000000020:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.791453+0000", 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 2.5993016, 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096136250000000006, 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2810 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791453+0000", 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791453+0000", 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791455+0000", 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0959999999999999e-06 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791463+0000", 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0860000000000002e-06 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791484+0000", 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0559000000000001e-05 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791492+0000", 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9300000000000003e-06 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791574+0000", 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2769000000000007e-05 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.791778+0000", 2024-09-16T11:13:28.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020306999999999999 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.793863+0000", 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020856780000000001 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.800911+0000", 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0070481780000000004 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.800958+0000", 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6523999999999997e-05 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.801019+0000", 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1438000000000004e-05 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.801067+0000", 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7296999999999998e-05 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.504 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:28.505 INFO:teuthology.orchestra.run.smithi162.stdout:696 trunc from 0xd855380 to 0x7b43d8a 2024-09-16T11:13:28.505 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:28.506 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:28.507 INFO:teuthology.orchestra.run.smithi162.stdout:697 punch from 0x5a8fcde to 0x5a9f79f, (0xfac1 bytes) 2024-09-16T11:13:28.507 INFO:teuthology.orchestra.run.smithi162.stdout:698 write 0x62f5828 thru 0x62f84ae (0x2c87 bytes) 2024-09-16T11:13:28.507 INFO:teuthology.orchestra.run.smithi162.stdout:699 writesame 0x2aaf84e thru 0x2ab9248 data_size 0x1(0x99fb bytes) 2024-09-16T11:13:28.507 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:28.507 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:28.508 INFO:teuthology.orchestra.run.smithi162.stdout:700 read 0x5fca893 thru 0x5fd22c9 (0x7a37 bytes) 2024-09-16T11:13:28.508 INFO:teuthology.orchestra.run.smithi162.stdout:701 punch from 0x240e2cc to 0x241574b, (0x747f bytes) 2024-09-16T11:13:28.508 INFO:teuthology.orchestra.run.smithi162.stdout:702 write 0x37dc7e3 thru 0x37ebee2 (0xf700 bytes) 2024-09-16T11:13:28.508 INFO:teuthology.orchestra.run.smithi162.stdout:703 read 0x322dd36 thru 0x3231f99 (0x4264 bytes) 2024-09-16T11:13:28.508 INFO:teuthology.orchestra.run.smithi162.stdout:704 punch from 0x28b73db to 0x28bb2ea, (0x3f0f bytes) 2024-09-16T11:13:28.509 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.509 INFO:teuthology.orchestra.run.smithi162.stdout:707 read 0xb483b thru 0xc23b3 (0xdb79 bytes) 2024-09-16T11:13:28.509 INFO:teuthology.orchestra.run.smithi162.stdout:709 write 0x26089bd thru 0x260902c (0x670 bytes) 2024-09-16T11:13:28.509 INFO:teuthology.orchestra.run.smithi162.stdout:710 read 0x5c06e64 thru 0x5c15e48 (0xefe5 bytes) 2024-09-16T11:13:28.509 INFO:teuthology.orchestra.run.smithi162.stdout:711 read 0x6d7f5a0 thru 0x6d88109 (0x8b6a bytes) 2024-09-16T11:13:28.509 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:28.510 INFO:teuthology.orchestra.run.smithi162.stdout:712 write 0xbabe331 thru 0xbac1cf8 (0x39c8 bytes) 2024-09-16T11:13:28.510 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.510 INFO:teuthology.orchestra.run.smithi162.stdout:714 read 0xaa36890 thru 0xaa4001d (0x978e bytes) 2024-09-16T11:13:28.510 INFO:teuthology.orchestra.run.smithi162.stdout:715 punch from 0x3e6daf3 to 0x3e7cb84, (0xf091 bytes) 2024-09-16T11:13:28.510 INFO:teuthology.orchestra.run.smithi162.stdout:716 write 0x98ca084 thru 0x98d88c7 (0xe844 bytes) 2024-09-16T11:13:28.510 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stdout:719 write 0x422701d thru 0x422fdb8 (0x8d9c bytes) 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:28.511 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:28.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:28.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:28.512 INFO:teuthology.orchestra.run.smithi162.stdout:722 writesame 0x9a35c96 thru 0x9a44bac data_size 0x1(0xef17 bytes) 2024-09-16T11:13:28.513 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:28.501+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:28.543 INFO:teuthology.orchestra.run.smithi162.stdout:723 write 0xd6e6895 thru 0xd6ef62f (0x8d9b bytes) 2024-09-16T11:13:28.549 INFO:teuthology.orchestra.run.smithi162.stdout:724 writesame 0xd47968d thru 0xd483131 data_size 0xb(0x9aa5 bytes) 2024-09-16T11:13:28.559 INFO:teuthology.orchestra.run.smithi162.stdout:725 write 0x40d926a thru 0x40e6f3f (0xdcd6 bytes) 2024-09-16T11:13:28.567 INFO:teuthology.orchestra.run.smithi162.stdout:732 read 0xbf7a464 thru 0xbf7eefe (0x4a9b bytes) 2024-09-16T11:13:28.567 INFO:teuthology.orchestra.run.smithi162.stdout:733 write 0x9e8e14d thru 0x9e940af (0x5f63 bytes) 2024-09-16T11:13:28.573 INFO:teuthology.orchestra.run.smithi162.stdout:735 read 0x29f7995 thru 0x29f8e8e (0x14fa bytes) 2024-09-16T11:13:28.573 INFO:teuthology.orchestra.run.smithi162.stdout:736 read 0x2a1b6b thru 0x2ad7bc (0xbc52 bytes) 2024-09-16T11:13:28.582 INFO:teuthology.orchestra.run.smithi162.stdout:737 write 0xd90f6a1 thru 0xd9158cd (0x622d bytes) 2024-09-16T11:13:28.600 INFO:teuthology.orchestra.run.smithi162.stdout:738 write 0xdcc5fd5 thru 0xdcc7479 (0x14a5 bytes) 2024-09-16T11:13:28.603 INFO:teuthology.orchestra.run.smithi162.stdout:740 read 0x8f18e76 thru 0x8f1b61d (0x27a8 bytes) 2024-09-16T11:13:28.604 INFO:teuthology.orchestra.run.smithi162.stdout:742 writesame 0x567fe1 thru 0x5702d2 data_size 0x6f(0x82f2 bytes) 2024-09-16T11:13:28.606 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:28.601+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:28.606 INFO:teuthology.orchestra.run.smithi162.stdout:743 read 0xbac4b4a thru 0xbad4099 (0xf550 bytes) 2024-09-16T11:13:28.607 INFO:teuthology.orchestra.run.smithi162.stdout:744 read 0xd2ae28a thru 0xd2b64c2 (0x8239 bytes) 2024-09-16T11:13:28.608 INFO:teuthology.orchestra.run.smithi162.stdout:745 read 0x2f5f9ed thru 0x2f6651e (0x6b32 bytes) 2024-09-16T11:13:28.608 INFO:teuthology.orchestra.run.smithi162.stdout:746 write 0xc1129a thru 0xc18a2f (0x7796 bytes) 2024-09-16T11:13:28.611 INFO:teuthology.orchestra.run.smithi162.stdout:749 read 0xb10f64e thru 0xb111ce9 (0x269c bytes) 2024-09-16T11:13:28.611 INFO:teuthology.orchestra.run.smithi162.stdout:750 punch from 0x53db0b7 to 0x53e123f, (0x6188 bytes) 2024-09-16T11:13:28.611 INFO:teuthology.orchestra.run.smithi162.stdout:752 write 0xb002201 thru 0xb00e9d5 (0xc7d5 bytes) 2024-09-16T11:13:28.618 INFO:teuthology.orchestra.run.smithi162.stdout:754 read 0x2569475 thru 0x2570510 (0x709c bytes) 2024-09-16T11:13:28.618 INFO:teuthology.orchestra.run.smithi162.stdout:755 read 0x54b709 thru 0x550102 (0x49fa bytes) 2024-09-16T11:13:28.619 INFO:teuthology.orchestra.run.smithi162.stdout:756 write 0x69bf890 thru 0x69cc875 (0xcfe6 bytes) 2024-09-16T11:13:28.629 INFO:teuthology.orchestra.run.smithi162.stdout:757 punch from 0x6d7e70d to 0x6d8722e, (0x8b21 bytes) 2024-09-16T11:13:28.629 INFO:teuthology.orchestra.run.smithi162.stdout:759 writesame 0x5fe7ae6 thru 0x5ff5498 data_size 0xd(0xd9b3 bytes) 2024-09-16T11:13:28.639 INFO:teuthology.orchestra.run.smithi162.stdout:760 writesame 0x29c27c5 thru 0x29cfc63 data_size 0xd(0xd49f bytes) 2024-09-16T11:13:28.646 INFO:teuthology.orchestra.run.smithi162.stdout:761 punch from 0x3921e97 to 0x3927869, (0x59d2 bytes) 2024-09-16T11:13:28.646 INFO:teuthology.orchestra.run.smithi162.stdout:762 punch from 0x7b5c361 to 0x7b6a3d0, (0xe06f bytes) 2024-09-16T11:13:28.661 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:28.661 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:28.664 INFO:teuthology.orchestra.run.smithi162.stdout:763 trunc from 0xdcc747a to 0x276e3ac 2024-09-16T11:13:28.675 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:28.692 INFO:teuthology.orchestra.run.smithi162.stdout:765 read 0x70d39 thru 0x7d0ed (0xc3b5 bytes) 2024-09-16T11:13:28.712 INFO:teuthology.orchestra.run.smithi162.stdout:766 writesame 0x7d46790 thru 0x7d4b2fa data_size 0x1(0x4b6b bytes) 2024-09-16T11:13:28.713 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:28.709+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:28.719 INFO:teuthology.orchestra.run.smithi162.stdout:767 trunc from 0x7d4b2fb to 0xbc5b6c3 2024-09-16T11:13:28.743 INFO:teuthology.orchestra.run.smithi162.stdout:768 writesame 0xd920ad3 thru 0xd930887 data_size 0x1(0xfdb5 bytes) 2024-09-16T11:13:28.743 INFO:teuthology.orchestra.run.smithi162.stdout:769 write 0x932e50b thru 0x93385d8 (0xa0ce bytes) 2024-09-16T11:13:28.749 INFO:teuthology.orchestra.run.smithi162.stdout:771 read 0x5eff69d thru 0x5f0cdfc (0xd760 bytes) 2024-09-16T11:13:28.749 INFO:teuthology.orchestra.run.smithi162.stdout:772 read 0xbe949b0 thru 0xbe9b0a5 (0x66f6 bytes) 2024-09-16T11:13:28.749 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.750 INFO:teuthology.orchestra.run.smithi162.stdout:777 writesame 0x2f65703 thru 0x2f6c4a7 data_size 0x1(0x6da5 bytes) 2024-09-16T11:13:28.792 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:28.792 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:28.792 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:28.793 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:28.793 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:28.801 INFO:teuthology.orchestra.run.smithi162.stdout:779 trunc from 0xd930888 to 0x23b61d6 2024-09-16T11:13:28.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:28.836 INFO:teuthology.orchestra.run.smithi162.stdout:780 write 0x4e9e54e thru 0x4ea4a0d (0x64c0 bytes) 2024-09-16T11:13:28.836 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:28.829+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:28.845 INFO:teuthology.orchestra.run.smithi162.stdout:781 read 0x45e575d thru 0x45ea8c9 (0x516d bytes) 2024-09-16T11:13:28.868 INFO:teuthology.orchestra.run.smithi162.stdout:782 write 0xbb56e5f thru 0xbb57d5e (0xf00 bytes) 2024-09-16T11:13:28.889 INFO:teuthology.orchestra.run.smithi162.stdout:784 write 0xd3196a8 thru 0xd320bbe (0x7517 bytes) 2024-09-16T11:13:28.897 INFO:teuthology.orchestra.run.smithi162.stdout:787 punch from 0x716beb5 to 0x7174237, (0x8382 bytes) 2024-09-16T11:13:28.897 INFO:teuthology.orchestra.run.smithi162.stdout:788 flatten 2024-09-16T11:13:28.907 INFO:teuthology.orchestra.run.smithi162.stdout:789 write 0x92207b7 thru 0x9228d02 (0x854c bytes) 2024-09-16T11:13:28.908 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:28.905+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:28.914 INFO:teuthology.orchestra.run.smithi162.stdout:791 read 0x9b01f43 thru 0x9b0354b (0x1609 bytes) 2024-09-16T11:13:28.915 INFO:teuthology.orchestra.run.smithi162.stdout:792 read 0xa00b3a0 thru 0xa0154bf (0xa120 bytes) 2024-09-16T11:13:28.915 INFO:teuthology.orchestra.run.smithi162.stdout:793 write 0xa72b802 thru 0xa731226 (0x5a25 bytes) 2024-09-16T11:13:28.920 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.920 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.920 INFO:teuthology.orchestra.run.smithi162.stdout:796 writesame 0xce9b8dc thru 0xcea2fbb data_size 0x30(0x76e0 bytes) 2024-09-16T11:13:28.930 INFO:teuthology.orchestra.run.smithi162.stdout:798 punch from 0xca1b3ba to 0xca25453, (0xa099 bytes) 2024-09-16T11:13:28.930 INFO:teuthology.orchestra.run.smithi162.stdout:800 read 0x1242ef1 thru 0x124dd58 (0xae68 bytes) 2024-09-16T11:13:28.930 INFO:teuthology.orchestra.run.smithi162.stdout:801 write 0x9d0203d thru 0x9d0c89b (0xa85f bytes) 2024-09-16T11:13:28.936 INFO:teuthology.orchestra.run.smithi162.stdout:802 write 0x8c9714e thru 0x8ca4f36 (0xdde9 bytes) 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:28.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.111759206999999, 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:28.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:28.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.821184436999999, 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:28.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:28.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:28.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:28.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.812776148999999, 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:28.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:28.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:28.959 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:803 read 0xb25f890 thru 0xb26884a (0x8fbb bytes) 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:805 punch from 0x2bc25d4 to 0x2bcece3, (0xc70f bytes) 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:806 read 0x66ee24e thru 0x66f6044 (0x7df7 bytes) 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:812 punch from 0xaed8a33 to 0xaed98a7, (0xe74 bytes) 2024-09-16T11:13:28.960 INFO:teuthology.orchestra.run.smithi162.stdout:813 write 0xcab5e0b thru 0xcabad2e (0x4f24 bytes) 2024-09-16T11:13:28.961 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:28.969 INFO:teuthology.orchestra.run.smithi162.stdout:814 trunc from 0xd320bbf to 0x1eda41d 2024-09-16T11:13:28.990 INFO:teuthology.orchestra.run.smithi162.stdout:815 write 0x1510913 thru 0x1518d14 (0x8402 bytes) 2024-09-16T11:13:29.009 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:29.005+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:29.020 INFO:teuthology.orchestra.run.smithi162.stdout:816 write 0xbddb7eb thru 0xbde24aa (0x6cc0 bytes) 2024-09-16T11:13:29.029 INFO:teuthology.orchestra.run.smithi162.stdout:817 write 0xa78c50d thru 0xa78f66c (0x3160 bytes) 2024-09-16T11:13:29.035 INFO:teuthology.orchestra.run.smithi162.stdout:818 read 0x46fb0f3 thru 0x46fbcf3 (0xc01 bytes) 2024-09-16T11:13:29.036 INFO:teuthology.orchestra.run.smithi162.stdout:819 trunc from 0xbde24ab to 0xcaa89ca 2024-09-16T11:13:29.045 INFO:teuthology.orchestra.run.smithi162.stdout:820 write 0x4f587a3 thru 0x4f6599a (0xd1f8 bytes) 2024-09-16T11:13:29.054 INFO:teuthology.orchestra.run.smithi162.stdout:821 read 0x9485f2e thru 0x948d1b8 (0x728b bytes) 2024-09-16T11:13:29.054 INFO:teuthology.orchestra.run.smithi162.stdout:822 punch from 0x40a18c to 0x41089a, (0x670e bytes) 2024-09-16T11:13:29.054 INFO:teuthology.orchestra.run.smithi162.stdout:823 read 0x80d66b7 thru 0x80d7b63 (0x14ad bytes) 2024-09-16T11:13:29.054 INFO:teuthology.orchestra.run.smithi162.stdout:824 write 0x400139b thru 0x400304f (0x1cb5 bytes) 2024-09-16T11:13:29.060 INFO:teuthology.orchestra.run.smithi162.stdout:825 write 0xc97904e thru 0xc97bb5a (0x2b0d bytes) 2024-09-16T11:13:29.066 INFO:teuthology.orchestra.run.smithi162.stdout:826 write 0x684c0e0 thru 0x684cf40 (0xe61 bytes) 2024-09-16T11:13:29.070 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:29.070 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:29.070 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:29.072 INFO:teuthology.orchestra.run.smithi162.stdout:828 writesame 0x761c473 thru 0x7620be8 data_size 0x3(0x4776 bytes) 2024-09-16T11:13:29.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:29.082 INFO:teuthology.orchestra.run.smithi162.stdout:829 read 0x394af77 thru 0x3959367 (0xe3f1 bytes) 2024-09-16T11:13:29.085 INFO:teuthology.orchestra.run.smithi162.stdout:830 trunc from 0xcaa89ca to 0xb1a03a1 2024-09-16T11:13:29.103 INFO:teuthology.orchestra.run.smithi162.stdout:833 writesame 0x36adb7d thru 0x36ba7f7 data_size 0x1(0xcc7b bytes) 2024-09-16T11:13:29.109 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:29.105+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:29.140 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0xe051743 2024-09-16T11:13:29.146 INFO:teuthology.orchestra.run.smithi162.stdout:835 trunc from 0xb1a03a1 to 0xe051743 2024-09-16T11:13:29.155 INFO:teuthology.orchestra.run.smithi162.stdout:836 writesame 0x40cac0a thru 0x40d4ec1 data_size 0x29(0xa2b8 bytes) 2024-09-16T11:13:29.158 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:29.159 INFO:teuthology.orchestra.run.smithi162.stdout:842 writesame 0xaea2acc thru 0xaea4f02 data_size 0x1(0x2437 bytes) 2024-09-16T11:13:29.178 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:29.179 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:29.189 INFO:teuthology.orchestra.run.smithi162.stdout:843 trunc from 0xe051743 to 0x61d223e 2024-09-16T11:13:29.192 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:29.208 INFO:teuthology.orchestra.run.smithi162.stdout:844 clone 6 order 20 su 4096 sc 14 2024-09-16T11:13:29.253 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:29.249+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:29.304 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:309 4.11 4:8e779da3:::rbd_object_map.11b5175acf49:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:19.165041+0000", 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.135175103, 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0051224349999999998, 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 309 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165041+0000", 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165041+0000", 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165043+0000", 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5280000000000002e-06 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165053+0000", 2024-09-16T11:13:29.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2329999999999995e-06 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165070+0000", 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7220000000000001e-05 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165078+0000", 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1980000000000001e-06 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165141+0000", 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2558999999999995e-05 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165221+0000", 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0735999999999994e-05 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.165427+0000", 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020601800000000001 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.168001+0000", 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025736380000000001 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.170071+0000", 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020705419999999999 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.170111+0000", 2024-09-16T11:13:29.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9242999999999997e-05 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.170163+0000", 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2519999999999999e-05 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:342 4.11 4:8d8f2f3d:::rbd_header.11b52605a025:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:19.205363+0000", 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.094852602, 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054140129999999996, 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 342 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.312 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205363+0000", 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205363+0000", 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205366+0000", 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205377+0000", 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1674e-05 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205404+0000", 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7064999999999999e-05 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205415+0000", 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0651e-05 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205507+0000", 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1724000000000005e-05 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.205735+0000", 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228703 2024-09-16T11:13:29.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.206076+0000", 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034055799999999997 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.207077+0000", 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001001346 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.210593+0000", 2024-09-16T11:13:29.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003515462 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.210672+0000", 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8978000000000002e-05 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.210777+0000", 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105549 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:358 4.11 4:8d8f2f3d:::rbd_header.11b52605a025:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:19.234772+0000", 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.065444081000001, 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054910059999999997, 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 358 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.318 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234772+0000", 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234772+0000", 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234774+0000", 2024-09-16T11:13:29.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.846e-06 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234782+0000", 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8049999999999996e-06 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234807+0000", 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4922999999999998e-05 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234832+0000", 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4742000000000001e-05 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.234920+0000", 2024-09-16T11:13:29.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7398999999999994e-05 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.235060+0000", 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000140603 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.235487+0000", 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042651000000000001 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.236508+0000", 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001021374 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.240102+0000", 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003594224 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.240185+0000", 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2405999999999997e-05 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:19.240263+0000", 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8174000000000001e-05 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:455 4.11 4:8d8f2f3d:::rbd_header.11b52605a025:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.153764+0000", 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1464515780000006, 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064078449999999997, 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 455 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153764+0000", 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153764+0000", 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153767+0000", 2024-09-16T11:13:29.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7460000000000001e-06 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153779+0000", 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1844e-05 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153804+0000", 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5384e-05 2024-09-16T11:13:29.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153817+0000", 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2897999999999999e-05 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153942+0000", 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124836 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.153953+0000", 2024-09-16T11:13:29.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0763e-05 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.156926+0000", 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029729230000000001 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.157126+0000", 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200313 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.157837+0000", 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071147500000000004 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.158820+0000", 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00098246399999999995 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.160084+0000", 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012641340000000001 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.160140+0000", 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5865000000000001e-05 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.160172+0000", 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2199999999999997e-05 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:469 4.11 4:8d8f2f3d:::rbd_header.11b52605a025:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.178484+0000", 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1217320720000004, 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053458530000000002, 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 469 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178484+0000", 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178484+0000", 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178485+0000", 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4959999999999999e-06 2024-09-16T11:13:29.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178498+0000", 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2408e-05 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178514+0000", 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6045e-05 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178522+0000", 2024-09-16T11:13:29.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6549999999999993e-06 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178620+0000", 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7492999999999996e-05 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.178761+0000", 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014102299999999999 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.179181+0000", 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042025299999999999 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.180266+0000", 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010845449999999999 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.183729+0000", 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0034629050000000001 2024-09-16T11:13:29.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.183791+0000", 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2125000000000006e-05 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.183830+0000", 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8905000000000001e-05 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:608 4.17 4:e936c02e:::rbd_data.11b52605a025.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.310018+0000", 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.9901975380000003, 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084401349999999997, 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 608 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310018+0000", 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310018+0000", 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310020+0000", 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0530000000000001e-06 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310172+0000", 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015145800000000001 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310209+0000", 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7382000000000002e-05 2024-09-16T11:13:29.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310220+0000", 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0597999999999999e-05 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310286+0000", 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5951000000000006e-05 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310513+0000", 2024-09-16T11:13:29.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022743700000000001 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310798+0000", 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028501999999999998 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.313622+0000", 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028238080000000001 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.318336+0000", 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047142190000000004 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.318416+0000", 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9849000000000001e-05 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.318458+0000", 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2360000000000001e-05 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:609 4.17 4:e936c02e:::rbd_data.11b52605a025.0000000000000000:head [truncate 9961472] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.310240+0000", 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.9899755789999993, 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013678967, 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 609 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310240+0000", 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310240+0000", 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310241+0000", 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.11e-06 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310246+0000", 2024-09-16T11:13:29.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310258+0000", 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2442999999999999e-05 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.310275+0000", 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6248000000000001e-05 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.312084+0000", 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018095609999999999 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.312174+0000", 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9703000000000005e-05 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.312417+0000", 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024304200000000001 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.314231+0000", 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001813973 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.323776+0000", 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0095453320000000001 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.323880+0000", 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104044 2024-09-16T11:13:29.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.323919+0000", 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8970999999999997e-05 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:875 4.17 4:ee4c7774:::rbd_data.11b585795d6b.0000000000000006:head [stat,zero 4765891~12011325] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.919471+0000", 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.3807449080000005, 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060377089999999996, 2024-09-16T11:13:29.350 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 875 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919471+0000", 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919471+0000", 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919471+0000", 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.39e-07 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919473+0000", 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919479+0000", 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.139e-06 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919483+0000", 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5429999999999998e-06 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.923375+0000", 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038920059999999999 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.923556+0000", 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018069599999999999 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.925509+0000", 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019530960000000001 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:878 4.14 4:2ea1db83:::rbd_data.11b585795d6b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.919514+0000", 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.380702264, 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050377579999999998, 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.355 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 878 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919514+0000", 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919514+0000", 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919514+0000", 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919516+0000", 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919522+0000", 2024-09-16T11:13:29.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.806e-06 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.919525+0000", 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9790000000000002e-06 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.920721+0000", 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011958660000000001 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.921182+0000", 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000461469 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.924551+0000", 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033692420000000002 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:928 4.1b e39/31 4:d9c065bc:::rbd_object_map.11b585795d6b:head v 39'23, mlcod=39'19)", 2024-09-16T11:13:29.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.999724+0000", 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.3004923000000002, 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050697260000000001, 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999724+0000", 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999724+0000", 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999725+0000", 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.184e-06 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999735+0000", 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604e-05 2024-09-16T11:13:29.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999762+0000", 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6701000000000001e-05 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999773+0000", 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0536e-05 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999941+0000", 2024-09-16T11:13:29.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016815299999999999 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.999976+0000", 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5502e-05 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:21.004693+0000", 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00471712 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:21.004793+0000", 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9926000000000005e-05 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1817 4.1d 4:bdb53122:::rbd_data.11b51c0df580.000000000000000c:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.312647+0000", 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.987568424, 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0074693490000000001, 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1817 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312647+0000", 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312647+0000", 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312650+0000", 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3800000000000001e-06 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312660+0000", 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7270000000000002e-06 2024-09-16T11:13:29.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312690+0000", 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0215000000000001e-05 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312705+0000", 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5616000000000001e-05 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312782+0000", 2024-09-16T11:13:29.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6852000000000002e-05 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312904+0000", 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012131899999999999 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313163+0000", 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025973199999999999 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.317811+0000", 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0046478759999999996 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319998+0000", 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021866770000000002 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320074+0000", 2024-09-16T11:13:29.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5794e-05 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320117+0000", 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3161000000000002e-05 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1819 4.1d 4:ba69c8d3:::rbd_data.11b51c0df580.000000000000000f:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.312783+0000", 2024-09-16T11:13:29.370 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.9874330770000004, 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0096970819999999992, 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1819 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312783+0000", 2024-09-16T11:13:29.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312783+0000", 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312784+0000", 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0500000000000002e-07 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312791+0000", 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312803+0000", 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2697000000000001e-05 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.312811+0000", 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0330000000000008e-06 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.314975+0000", 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021640140000000001 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315067+0000", 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1403999999999994e-05 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.315294+0000", 2024-09-16T11:13:29.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00022665200000000001 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319964+0000", 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0046705549999999998 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.322378+0000", 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024140160000000002 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.322436+0000", 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7484000000000001e-05 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.322480+0000", 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4304999999999997e-05 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:1816 4.f e41/31 4:f1d27fab:::rbd_data.11b51c0df580.000000000000000a:head v 41'11, mlcod=40'8)", 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.313614+0000", 2024-09-16T11:13:29.376 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.9866014960000005, 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0064578580000000003, 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313614+0000", 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313614+0000", 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313616+0000", 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2130000000000001e-06 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313627+0000", 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1386000000000001e-05 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313645+0000", 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8462999999999999e-05 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.313672+0000", 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6353999999999999e-05 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.314029+0000", 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035763 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.314069+0000", 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9728000000000003e-05 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.319998+0000", 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0059289319999999996 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.320072+0000", 2024-09-16T11:13:29.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4152000000000004e-05 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:1825 4.1d 4:ba00c444:::rbd_data.11b51c0df580.0000000000000004:head [stat,truncate 1048576] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:24.328854+0000", 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.9713621689999998, 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0084784609999999996, 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 1825 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328854+0000", 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328854+0000", 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328855+0000", 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.725e-06 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328862+0000", 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.985e-06 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328882+0000", 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9273e-05 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328890+0000", 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1519999999999992e-06 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.328962+0000", 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1829000000000006e-05 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329066+0000", 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104027 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.329270+0000", 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020430000000000001 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.332121+0000", 2024-09-16T11:13:29.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028513900000000001 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337234+0000", 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.005113088 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337289+0000", 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4398000000000003e-05 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:24.337332+0000", 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3294000000000003e-05 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:3875 4.17 4:ebebcad9:::rbd_data.11b5b93336d2.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.79661357499999996, 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02646399, 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.387 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3875 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503604+0000", 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503618+0000", 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.375e-05 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503642+0000", 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.357e-05 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503651+0000", 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1060000000000006e-06 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503714+0000", 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3368999999999996e-05 2024-09-16T11:13:29.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503856+0000", 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014167699999999999 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.510013+0000", 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061567940000000002 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529819+0000", 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019806462 2024-09-16T11:13:29.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529932+0000", 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112724 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530010+0000", 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8412999999999996e-05 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530066+0000", 2024-09-16T11:13:29.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6004000000000001e-05 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:3875 4.17 e44/31 ondisk, result = 0)", 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.522324+0000", 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.77789210799999997, 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.007754113, 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.522324+0000", 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.522324+0000", 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.522325+0000", 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0580000000000001e-06 2024-09-16T11:13:29.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.522330+0000", 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7330000000000003e-06 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.522344+0000", 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4895999999999999e-05 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.522351+0000", 2024-09-16T11:13:29.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3759999999999996e-06 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529905+0000", 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0075540240000000003 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529919+0000", 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3967e-05 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530078+0000", 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000159059 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:3954 4.10 4:08836ffb:::rbd_data.11b5b93336d2.000000000000005f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.676507+0000", 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.62370858600000001, 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050714280000000002, 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3954 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676507+0000", 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676507+0000", 2024-09-16T11:13:29.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676510+0000", 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000002e-06 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676521+0000", 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1463e-05 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676557+0000", 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5323e-05 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676573+0000", 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6212999999999999e-05 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676621+0000", 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8482000000000001e-05 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676666+0000", 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4351999999999998e-05 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.676722+0000", 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6314999999999999e-05 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.679294+0000", 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025718160000000002 2024-09-16T11:13:29.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.681521+0000", 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022269730000000001 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.681549+0000", 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7818999999999999e-05 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.681579+0000", 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0212000000000001e-05 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:4086 4.1d 4:bd1e68ee:::rbd_data.11b5b93336d2.0000000000000053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.972721+0000", 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.32749497500000002, 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060453390000000003, 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 4086 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.972721+0000", 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.972721+0000", 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.972724+0000", 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0369999999999999e-06 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.972737+0000", 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3182e-05 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.972779+0000", 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1436000000000001e-05 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.972793+0000", 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4919999999999999e-05 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.973674+0000", 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00088028800000000001 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.973892+0000", 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000217918 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.974359+0000", 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000467756 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.976210+0000", 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0018508140000000001 2024-09-16T11:13:29.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.978677+0000", 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024671630000000001 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.978725+0000", 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7540999999999998e-05 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.978766+0000", 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1284e-05 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:4158 4.0 4:047b769a:::rbd_data.11b5b93336d2.000000000000001d:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.19374532699999999, 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032637175999999997, 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:29.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 4158 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:29.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106472+0000", 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.35e-06 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106479+0000", 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106496+0000", 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106503+0000", 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106568+0000", 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4346999999999999e-05 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106646+0000", 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8904000000000002e-05 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.114604+0000", 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079574510000000008 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.138952+0000", 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024348172000000001 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139028+0000", 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6315000000000004e-05 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139080+0000", 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1450999999999999e-05 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139108+0000", 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7855e-05 2024-09-16T11:13:29.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:4158 4.0 e44/31 ondisk, result = 0)", 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:29.127555+0000", 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.172660444, 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011404961, 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.127555+0000", 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.127555+0000", 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.127557+0000", 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5600000000000001e-06 2024-09-16T11:13:29.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.127564+0000", 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9770000000000003e-06 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.127586+0000", 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1668999999999999e-05 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.127596+0000", 2024-09-16T11:13:29.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0538e-05 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.138932+0000", 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011336264 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.138943+0000", 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0404999999999999e-05 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:29.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.138960+0000", 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7547999999999999e-05 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:29.419 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:29.421 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:29.421 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:29.417+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:29.521 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:29.517+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:29.528 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:29.528 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:29.528 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:29.528 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:29.537 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:29.623 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:29.617+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:29.634 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:29.634 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:29.643 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:29.724 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:29.721+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:29.746 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:9 4.1a 4:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.633879186, 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080919660000000008, 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.747 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 9 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108611+0000", 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4970000000000001e-06 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108629+0000", 2024-09-16T11:13:29.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8527999999999999e-05 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108666+0000", 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6965000000000003e-05 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108686+0000", 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9457999999999999e-05 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108888+0000", 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020222999999999999 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108902+0000", 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4511999999999999e-05 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.112496+0000", 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035939050000000001 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.112726+0000", 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000229559 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.113104+0000", 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000377592 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.114998+0000", 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018945019999999999 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116382+0000", 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001384159 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116556+0000", 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000174289 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116699+0000", 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014276999999999999 2024-09-16T11:13:29.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:613 4.13 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.311926+0000", 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.4305609140000008, 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010525013, 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 613 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311926+0000", 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311926+0000", 2024-09-16T11:13:29.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311927+0000", 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.449e-06 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311953+0000", 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6435999999999999e-05 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311970+0000", 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7054e-05 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311990+0000", 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9157999999999999e-05 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312056+0000", 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6246000000000002e-05 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312224+0000", 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016762600000000001 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312442+0000", 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021867300000000001 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316530+0000", 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040882239999999997 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322252+0000", 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057218080000000001 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322398+0000", 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014619200000000001 2024-09-16T11:13:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322451+0000", 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2147000000000001e-05 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:614 4.13 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head [truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.312014+0000", 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.4304722759999997, 2024-09-16T11:13:29.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011819527, 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 614 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312014+0000", 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312014+0000", 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312015+0000", 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8299999999999996e-07 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312018+0000", 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312028+0000", 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312033+0000", 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5249999999999996e-06 2024-09-16T11:13:29.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312760+0000", 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000726759 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312820+0000", 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0334999999999997e-05 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:29.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312955+0000", 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000134411 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.319112+0000", 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061578800000000001 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323699+0000", 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045862710000000003 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323770+0000", 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0999000000000002e-05 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323834+0000", 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3930000000000006e-05 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:872 4.13 4:ca9f37af:::rbd_data.11b585795d6b.0000000000000003:head [stat,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.8254148309999998, 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010169020000000001, 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.767 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 872 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7999999999999995e-07 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917075+0000", 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.932e-06 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917085+0000", 2024-09-16T11:13:29.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0248000000000001e-05 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917091+0000", 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9569999999999999e-06 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.921551+0000", 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044594609999999996 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.921682+0000", 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013097 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.921849+0000", 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167348 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.922837+0000", 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098752400000000004 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.927113+0000", 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042765019999999997 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.927183+0000", 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9698999999999998e-05 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.927241+0000", 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7799e-05 2024-09-16T11:13:29.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:891 4.9 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 8388608~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.8134411440000004, 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015304566, 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.773 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 891 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929047+0000", 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4729999999999999e-06 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933394+0000", 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043469529999999998 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933425+0000", 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1238999999999997e-05 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933441+0000", 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5719999999999999e-05 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933522+0000", 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1339000000000002e-05 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933648+0000", 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000125883 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933908+0000", 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026008699999999999 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.942070+0000", 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0081615249999999993 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944253+0000", 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021831910000000001 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944310+0000", 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7241999999999998e-05 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944350+0000", 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9913999999999998e-05 2024-09-16T11:13:29.778 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:916 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984413+0000", 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7580732779999995, 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073160899999999999, 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.779 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 916 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984413+0000", 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984413+0000", 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984414+0000", 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5999999999999991e-07 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984421+0000", 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.832e-06 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984434+0000", 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3192e-05 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984442+0000", 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.986810+0000", 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023685989999999999 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.986918+0000", 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010765200000000001 2024-09-16T11:13:29.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.987391+0000", 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000473258 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989552+0000", 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021602890000000001 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991602+0000", 2024-09-16T11:13:29.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020509130000000001 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991661+0000", 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8295999999999999e-05 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991729+0000", 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8492999999999996e-05 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:917 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984451+0000", 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7580353590000009, 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0081242050000000007, 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 917 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984451+0000", 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984451+0000", 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984452+0000", 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2500000000000004e-07 2024-09-16T11:13:29.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984457+0000", 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7899999999999999e-06 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984473+0000", 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6183000000000001e-05 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984481+0000", 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.695e-06 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.988141+0000", 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036604089999999999 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.988234+0000", 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2855000000000005e-05 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.988642+0000", 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000407649 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991549+0000", 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0029075989999999999 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992507+0000", 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095825900000000004 2024-09-16T11:13:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992547+0000", 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9165999999999997e-05 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992575+0000", 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8674999999999999e-05 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:918 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984490+0000", 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7579966460000005, 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082302569999999995, 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 918 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984490+0000", 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984490+0000", 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984491+0000", 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984496+0000", 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0669999999999999e-06 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984509+0000", 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2736e-05 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984516+0000", 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6960000000000005e-06 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989623+0000", 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051062870000000002 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989734+0000", 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111074 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990137+0000", 2024-09-16T11:13:29.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040373900000000002 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991557+0000", 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001419897 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992620+0000", 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00106251 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992671+0000", 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1359000000000001e-05 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992720+0000", 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8881000000000002e-05 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:919 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984525+0000", 2024-09-16T11:13:29.796 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7579610310000007, 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094882379999999995, 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 919 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984525+0000", 2024-09-16T11:13:29.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984525+0000", 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984526+0000", 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2399999999999997e-07 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984531+0000", 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984567+0000", 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984575+0000", 2024-09-16T11:13:29.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3389999999999993e-06 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990760+0000", 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061848299999999997 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990836+0000", 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6500000000000003e-05 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991213+0000", 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037719499999999998 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992237+0000", 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001024016 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.993619+0000", 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001381267 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.993816+0000", 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019750799999999999 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.994014+0000", 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000197395 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:920 4.9 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.989540+0000", 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7529461659999992, 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076473119999999999, 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 920 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989540+0000", 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989540+0000", 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989543+0000", 2024-09-16T11:13:29.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3530000000000002e-06 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989555+0000", 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2646999999999999e-05 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989578+0000", 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2603999999999999e-05 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989591+0000", 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2984e-05 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989706+0000", 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011489699999999999 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989892+0000", 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000186465 2024-09-16T11:13:29.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990156+0000", 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026365600000000002 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.995724+0000", 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005567944 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997072+0000", 2024-09-16T11:13:29.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001348506 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997134+0000", 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1396999999999995e-05 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997188+0000", 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3859000000000001e-05 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:930 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7451878890000003, 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080905809999999995, 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 930 2024-09-16T11:13:29.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0100000000000004e-07 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997303+0000", 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2559999999999998e-06 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997313+0000", 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0434e-05 2024-09-16T11:13:29.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997317+0000", 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1840000000000001e-06 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.999225+0000", 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001907883 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.999310+0000", 2024-09-16T11:13:29.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4845999999999994e-05 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.999700+0000", 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00039045900000000001 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.003793+0000", 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040925340000000001 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.005311+0000", 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001518482 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.005357+0000", 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5621999999999998e-05 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.005389+0000", 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2079999999999998e-05 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:1000 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:21.177824+0000", 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.5646626920000006, 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010724525, 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1000 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177824+0000", 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177824+0000", 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177825+0000", 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1969999999999999e-06 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177831+0000", 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.353e-06 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177848+0000", 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6211000000000001e-05 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177857+0000", 2024-09-16T11:13:29.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9569999999999997e-06 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.182271+0000", 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044136790000000002 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.182398+0000", 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012694400000000001 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.182908+0000", 2024-09-16T11:13:29.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000509667 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.184150+0000", 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00124249 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188479+0000", 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043282540000000001 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188511+0000", 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2466999999999999e-05 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188548+0000", 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7305999999999998e-05 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:1006 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:29.820 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:21.188332+0000", 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.5541543920000009, 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074913419999999998, 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1006 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188332+0000", 2024-09-16T11:13:29.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188332+0000", 2024-09-16T11:13:29.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188333+0000", 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188336+0000", 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1609999999999999e-06 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188360+0000", 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-05 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188364+0000", 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0239999999999999e-06 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.189849+0000", 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014850429999999999 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.189907+0000", 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8303000000000001e-05 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.190209+0000", 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301947 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.191731+0000", 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001521957 2024-09-16T11:13:29.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.195761+0000", 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040294659999999998 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.195797+0000", 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5918999999999999e-05 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.195823+0000", 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6659999999999999e-05 2024-09-16T11:13:29.827 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2095 4.1b 4:ddcc03d7:::rbd_data.11b51c0df580.0000000000000000:7 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.526746+0000", 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2157404170000001, 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007442387, 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2095 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526746+0000", 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526746+0000", 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526747+0000", 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9999999999999997e-07 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526749+0000", 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7530000000000002e-06 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526758+0000", 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0770000000000008e-06 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526763+0000", 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1100000000000002e-06 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.533931+0000", 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0071678949999999996 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534006+0000", 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5722000000000004e-05 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534188+0000", 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018212999999999999 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2096 4.1b 4:ddcc03d7:::rbd_data.11b51c0df580.0000000000000000:7 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.526768+0000", 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.21571842, 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076987260000000004, 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2096 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526768+0000", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526768+0000", 2024-09-16T11:13:29.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526769+0000", 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999998e-07 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526771+0000", 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5399999999999998e-06 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526779+0000", 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0970000000000006e-06 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526785+0000", 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2109999999999997e-06 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534312+0000", 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075269769999999998 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534373+0000", 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1636000000000004e-05 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534467+0000", 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3561999999999996e-05 2024-09-16T11:13:29.836 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2097 4.1b 4:ddcc03d7:::rbd_data.11b51c0df580.0000000000000000:7 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.526790+0000", 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2156965289999997, 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078954669999999998, 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.837 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2097 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526790+0000", 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526790+0000", 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526791+0000", 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0599999999999999e-07 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526793+0000", 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5940000000000001e-06 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526803+0000", 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8330000000000006e-06 2024-09-16T11:13:29.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526809+0000", 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7409999999999998e-06 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534503+0000", 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076938589999999999 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534585+0000", 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2422999999999993e-05 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534685+0000", 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100111 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2184 4.9 4:93045ea3:::rbd_data.11b5220ac4a2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:29.841 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.532860+0000", 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2096260149999996, 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098338330000000002, 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2184 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.532860+0000", 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.532860+0000", 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.532862+0000", 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.658e-06 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534381+0000", 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001519095 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534395+0000", 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4053000000000001e-05 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534405+0000", 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.012e-05 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535334+0000", 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00092820100000000003 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535496+0000", 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016220899999999999 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535870+0000", 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037407100000000002 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539197+0000", 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033272689999999999 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.542561+0000", 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033639360000000001 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.542641+0000", 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9744000000000001e-05 2024-09-16T11:13:29.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.542694+0000", 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3477000000000003e-05 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:3939 4.1a 4:59e28139:::rbd_data.11b5b93336d2.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:28.667389+0000", 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.075097392, 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080127329999999993, 2024-09-16T11:13:29.847 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 3939 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667389+0000", 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667389+0000", 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667391+0000", 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.323e-06 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667403+0000", 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1219999999999999e-05 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667435+0000", 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2564999999999997e-05 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667449+0000", 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3963e-05 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667548+0000", 2024-09-16T11:13:29.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8692999999999999e-05 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667684+0000", 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013601999999999999 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667846+0000", 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016197800000000001 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.669208+0000", 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001361952 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.675143+0000", 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059356290000000004 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.675283+0000", 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139937 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.675402+0000", 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118453 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:4001 4.1b 4:dba71535:::rbd_data.11b5b93336d2.0000000000000016:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.99106687199999999, 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.027255873, 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.853 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 4001 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:29.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751422+0000", 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2469999999999998e-06 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751434+0000", 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1761e-05 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751469+0000", 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5577000000000002e-05 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751483+0000", 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3570999999999999e-05 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751628+0000", 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014536400000000001 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751857+0000", 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000228452 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.758485+0000", 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066286000000000001 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778412+0000", 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019927027 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778513+0000", 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010084 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778611+0000", 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8432999999999998e-05 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778675+0000", 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4000999999999999e-05 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:4170 4.1 4:8514a423:::rbd_data.11b5b93336d2.0000000000000059:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:29.161302+0000", 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.58118399799999998, 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011719693999999999, 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 4170 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.859 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161302+0000", 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161302+0000", 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161305+0000", 2024-09-16T11:13:29.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.12e-06 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161316+0000", 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0937e-05 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161350+0000", 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4034e-05 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161362+0000", 2024-09-16T11:13:29.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2874000000000001e-05 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161481+0000", 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000119083 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161745+0000", 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000263906 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:29.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.163591+0000", 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001845675 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.172728+0000", 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091370110000000004 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.172858+0000", 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012993 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:29.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.172961+0000", 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102699 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.173022+0000", 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1425000000000002e-05 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:29.864 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:29.865 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:29.865 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:29.865 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T11:13:29.865 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:13:29.867 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:29.867 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:29.861+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:29.967 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:29.961+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:30.017 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:30.017 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:30.017 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:30.017 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:30.027 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:30.072 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:30.069+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:30.125 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:30.126 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:30.126 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:30.126 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:30.126 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:30.135 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:30.169 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:30.165+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:30.174 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 44, 2024-09-16T11:13:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7e3ff0c6-8090-4b11-95d6-fd611813de49", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2024-09-16T11:12:35.459871+0000", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2024-09-16T11:13:28.190048+0000", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2024-09-16T11:12:46.645944+0000", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2024-09-16T11:13:18.117664+0000", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5806080, 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "nodeep-scrub", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "noscrub", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2024-09-16T11:13:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 4, 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "quincy", 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": ".mgr", 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:48.004741+0000", 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "17", 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_max": 32, 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": {} 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:53.361644+0000", 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:13:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2024-09-16T11:13:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "26", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [ 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: 3 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": 3, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": 3, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 3, 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cache", 2024-09-16T11:13:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:02.524586+0000", 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8201, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 4, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 4, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 4, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 4, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 4, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "30", 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:13:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": 2, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "writeback", 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 250, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "bloom", 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "target_size": 0, 2024-09-16T11:13:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "seed": 0 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 60, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 8, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [ 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "pool_client.0", 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:11.876887+0000", 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:30.194 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 32, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 32, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 32, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 32, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 32, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "43", 2024-09-16T11:13:30.195 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 8, 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 43, 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b772f030-6c52-46c6-a992-36cadd7f9edd", 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2024-09-16T11:13:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2024-09-16T11:13:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6825/22248", 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6827/22248", 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6831/22248", 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6829/22248", 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6335aa8f-a82b-45a4-be04-b1b064314f8d", 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6800", 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6801", 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2024-09-16T11:13:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6801/22247", 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6803/22247", 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6807/22247", 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6805/22247", 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "8d9432da-1713-44e1-9c96-db33f21cf333", 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 0, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 0, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6817/22258", 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6819/22258", 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6823/22258", 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6821/22258", 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:13:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "f17d5292-6b43-4887-a146-a647c60e4f86", 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6809/22257", 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6811/22257", 2024-09-16T11:13:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6815/22257", 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6813/22257", 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "93cb4050-ba1d-44c4-8e05-854a4cbb9236", 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6800", 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6801", 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6802", 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6803", 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6806", 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6807", 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6804", 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6805", 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6801/22379", 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6803/22379", 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6807/22379", 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6805/22379", 2024-09-16T11:13:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "432a0446-4f3b-436d-bba0-76184260e13c", 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6808", 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6809", 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6810", 2024-09-16T11:13:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6811", 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6814", 2024-09-16T11:13:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6815", 2024-09-16T11:13:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6812", 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6813", 2024-09-16T11:13:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6809/22380", 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6811/22380", 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6815/22380", 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6813/22380", 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "7c7e65e2-ab06-47f4-8d6d-712aa308e13f", 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6816", 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6818", 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6819", 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6821", 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6826", 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6827", 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6822", 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6824", 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6818/22381", 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6821/22381", 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6827/22381", 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6824/22381", 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:13:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "827daf60-a01e-40a5-9595-7ec5406e4a8a", 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 18, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6817", 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6820", 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6823", 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6825", 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6830", 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6831", 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6828", 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6829", 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6820/22378", 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6825/22378", 2024-09-16T11:13:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6831/22378", 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6829/22378", 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.249 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.054827+0000", 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.474331+0000", 2024-09-16T11:13:30.250 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 65536, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.753279+0000", 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:13:30.251 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.733978+0000", 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.252 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.980000+0000", 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.142568+0000", 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.883591+0000", 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:30.254 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.129808+0000", 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklist": {}, 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "range_blocklist": {}, 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2024-09-16T11:13:30.255 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {}, 2024-09-16T11:13:30.256 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode": { 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_enabled": false, 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_bucket_count": 0, 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_bucket": 0 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:30.257 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:30.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:13:30.260 INFO:tasks.thrashosds.thrasher:j is {'epoch': 44, 'fsid': '7e3ff0c6-8090-4b11-95d6-fd611813de49', 'created': '2024-09-16T11:12:35.459871+0000', 'modified': '2024-09-16T11:13:28.190048+0000', 'last_up_change': '2024-09-16T11:12:46.645944+0000', 'last_in_change': '2024-09-16T11:13:18.117664+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:12:48.004741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:12:53.361644+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:13:02.524586+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:13:11.876887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 8, 'snap_epoch': 43, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'b772f030-6c52-46c6-a992-36cadd7f9edd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6825', 'nonce': 22248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6826', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 22248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6830', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 22248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6829', 'nonce': 22248}]}, 'public_addr': '172.21.15.162:6825/22248', 'cluster_addr': '172.21.15.162:6827/22248', 'heartbeat_back_addr': '172.21.15.162:6831/22248', 'heartbeat_front_addr': '172.21.15.162:6829/22248', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6335aa8f-a82b-45a4-be04-b1b064314f8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 22247}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 22247}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 22247}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 22247}]}, 'public_addr': '172.21.15.162:6801/22247', 'cluster_addr': '172.21.15.162:6803/22247', 'heartbeat_back_addr': '172.21.15.162:6807/22247', 'heartbeat_front_addr': '172.21.15.162:6805/22247', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8d9432da-1713-44e1-9c96-db33f21cf333', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6817', 'nonce': 22258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6818', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 22258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6822', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 22258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6821', 'nonce': 22258}]}, 'public_addr': '172.21.15.162:6817/22258', 'cluster_addr': '172.21.15.162:6819/22258', 'heartbeat_back_addr': '172.21.15.162:6823/22258', 'heartbeat_front_addr': '172.21.15.162:6821/22258', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f17d5292-6b43-4887-a146-a647c60e4f86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 22257}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 22257}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 22257}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 22257}]}, 'public_addr': '172.21.15.162:6809/22257', 'cluster_addr': '172.21.15.162:6811/22257', 'heartbeat_back_addr': '172.21.15.162:6815/22257', 'heartbeat_front_addr': '172.21.15.162:6813/22257', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '93cb4050-ba1d-44c4-8e05-854a4cbb9236', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 22379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 22379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 22379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 22379}]}, 'public_addr': '172.21.15.165:6801/22379', 'cluster_addr': '172.21.15.165:6803/22379', 'heartbeat_back_addr': '172.21.15.165:6807/22379', 'heartbeat_front_addr': '172.21.15.165:6805/22379', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '432a0446-4f3b-436d-bba0-76184260e13c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 22380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 22380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 22380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 22380}]}, 'public_addr': '172.21.15.165:6809/22380', 'cluster_addr': '172.21.15.165:6811/22380', 'heartbeat_back_addr': '172.21.15.165:6815/22380', 'heartbeat_front_addr': '172.21.15.165:6813/22380', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c7e65e2-ab06-47f4-8d6d-712aa308e13f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6818', 'nonce': 22381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6819', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 22381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 22381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6824', 'nonce': 22381}]}, 'public_addr': '172.21.15.165:6818/22381', 'cluster_addr': '172.21.15.165:6821/22381', 'heartbeat_back_addr': '172.21.15.165:6827/22381', 'heartbeat_front_addr': '172.21.15.165:6824/22381', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '827daf60-a01e-40a5-9595-7ec5406e4a8a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6817', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6820', 'nonce': 22378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6823', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 22378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 22378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 22378}]}, 'public_addr': '172.21.15.165:6820/22378', 'cluster_addr': '172.21.15.165:6825/22378', 'heartbeat_back_addr': '172.21.15.165:6831/22378', 'heartbeat_front_addr': '172.21.15.165:6829/22378', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.054827+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.474331+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.753279+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.733978+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.980000+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.142568+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.883591+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.129808+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:13:30.260 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 35.410419077999997, 2024-09-16T11:13:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091644129999999997, 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822531+0000", 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1739999999999999e-06 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822539+0000", 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4050000000000003e-06 2024-09-16T11:13:30.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822554+0000", 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5384e-05 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822564+0000", 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7510000000000003e-06 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822695+0000", 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131481 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822720+0000", 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5069999999999999e-05 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826480+0000", 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037597590000000001 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826680+0000", 2024-09-16T11:13:30.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000200304 2024-09-16T11:13:30.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 2", 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826846+0000", 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016532299999999999 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828757+0000", 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001911598 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831554+0000", 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027972930000000002 2024-09-16T11:13:30.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831613+0000", 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8715000000000003e-05 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831690+0000", 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7156000000000005e-05 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:594 4.8 4:125c4097:::rbd_object_map.11b52605a025.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301603+0000", 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.9313421010000003, 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0068584780000000003, 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 594 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.269 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301603+0000", 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301603+0000", 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301604+0000", 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:13:30.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301608+0000", 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4050000000000002e-06 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301616+0000", 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1040000000000007e-06 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301623+0000", 2024-09-16T11:13:30.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303230+0000", 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0016072149999999999 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303314+0000", 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3752999999999996e-05 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:30.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303660+0000", 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034616399999999999 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308254+0000", 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004593728 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308372+0000", 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117806 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308423+0000", 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.147e-05 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308461+0000", 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.841e-05 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.274 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:602 4.8 4:125c4097:::rbd_object_map.11b52605a025.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303307+0000", 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.9296378579999995, 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079281130000000005, 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 602 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303307+0000", 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303307+0000", 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303308+0000", 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0699999999999996e-07 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303313+0000", 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0470000000000001e-06 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303325+0000", 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1574e-05 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303332+0000", 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3039999999999999e-06 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307030+0000", 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036981510000000002 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307138+0000", 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010738000000000001 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307546+0000", 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040797799999999998 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311124+0000", 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035785550000000002 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311132+0000", 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9570000000000002e-06 2024-09-16T11:13:30.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311194+0000", 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2286999999999995e-05 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311235+0000", 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0973000000000002e-05 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:615 4.15 4:ad422a37:::rbd_data.11b52605a025.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.9210315139999992, 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010270354000000001, 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 615 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.281 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311915+0000", 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1969999999999999e-06 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311920+0000", 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0010000000000001e-06 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311933+0000", 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2948e-05 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311937+0000", 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8219999999999997e-06 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311999+0000", 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1174999999999996e-05 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312158+0000", 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015924900000000001 2024-09-16T11:13:30.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312292+0000", 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013398899999999999 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313867+0000", 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001575087 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322101+0000", 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082335659999999995 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322144+0000", 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3900000000000003e-05 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322184+0000", 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9419999999999999e-05 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3699912659999995, 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:13:30.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2048 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524324+0000", 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7086214389999999, 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070262839999999998, 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.291 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2048 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524324+0000", 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524324+0000", 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524325+0000", 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0419999999999999e-06 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524337+0000", 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2853999999999999e-05 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524351+0000", 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3552e-05 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524363+0000", 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531097+0000", 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006733859 2024-09-16T11:13:30.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531200+0000", 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102729 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531350+0000", 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015020499999999999 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2066 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524379+0000", 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7085662700000004, 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072794260000000003, 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2066 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.296 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524379+0000", 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524379+0000", 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524380+0000", 2024-09-16T11:13:30.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1200000000000001e-07 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524384+0000", 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5279999999999997e-06 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524397+0000", 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3083e-05 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524408+0000", 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1131000000000001e-05 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531392+0000", 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069834820000000001 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531522+0000", 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013043700000000001 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531658+0000", 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013585299999999999 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2067 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1572864~39333] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524422+0000", 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7085226740000001, 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098132879999999999, 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2067 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524422+0000", 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524422+0000", 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524423+0000", 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1999999999999998e-07 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524428+0000", 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5179999999999998e-06 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524441+0000", 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.348e-05 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524450+0000", 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4330000000000004e-06 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531701+0000", 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072510020000000003 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531814+0000", 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011318400000000001 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534236+0000", 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024217510000000002 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.304 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2068 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7084674819999996, 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099591620000000006, 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2068 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524479+0000", 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.155e-06 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524483+0000", 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5680000000000001e-06 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524497+0000", 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4018e-05 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524509+0000", 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1735e-05 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534308+0000", 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097988120000000005 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534360+0000", 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2094e-05 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534437+0000", 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6779999999999993e-05 2024-09-16T11:13:30.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2069 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7084194950000002, 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010200859, 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2069 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524527+0000", 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.02e-06 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524531+0000", 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4900000000000002e-06 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524544+0000", 2024-09-16T11:13:30.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.311e-05 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524552+0000", 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6879999999999999e-06 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534481+0000", 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099292119999999998 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534629+0000", 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148281 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534726+0000", 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7058000000000005e-05 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2081 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525000+0000", 2024-09-16T11:13:30.313 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7079445939999998, 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009893542, 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2081 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525000+0000", 2024-09-16T11:13:30.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525000+0000", 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525003+0000", 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8090000000000001e-06 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525014+0000", 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525040+0000", 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5925999999999999e-05 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525057+0000", 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6293000000000001e-05 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534751+0000", 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009694055 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534813+0000", 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2318999999999997e-05 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534894+0000", 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1057000000000007e-05 2024-09-16T11:13:30.317 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2082 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7078723919999996, 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010054116, 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2082 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525074+0000", 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4470000000000001e-06 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525080+0000", 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8980000000000001e-06 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525099+0000", 2024-09-16T11:13:30.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8960000000000001e-05 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525112+0000", 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3013999999999999e-05 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534937+0000", 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098250449999999993 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535013+0000", 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6108999999999999e-05 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535127+0000", 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113643 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2083 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.322 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.707818563, 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010171747, 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2083 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:30.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525128+0000", 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.082e-06 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525134+0000", 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8079999999999999e-06 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525149+0000", 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4833e-05 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525158+0000", 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5820000000000002e-06 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535171+0000", 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010012828999999999 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535224+0000", 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3826999999999998e-05 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535298+0000", 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3786000000000001e-05 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.326 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2098 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7077066319999998, 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010231951, 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2098 2024-09-16T11:13:30.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525240+0000", 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.494e-06 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525247+0000", 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7710000000000001e-06 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525265+0000", 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7966e-05 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525276+0000", 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1827e-05 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535332+0000", 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010055633 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535388+0000", 2024-09-16T11:13:30.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5741e-05 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535470+0000", 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2519000000000001e-05 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2099 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7074925700000003, 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010173994, 2024-09-16T11:13:30.331 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2099 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525455+0000", 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6469999999999998e-06 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525468+0000", 2024-09-16T11:13:30.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2931e-05 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525518+0000", 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0448000000000002e-05 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525542+0000", 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3870999999999999e-05 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535510+0000", 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099681400000000003 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535559+0000", 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8028999999999997e-05 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535626+0000", 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7928000000000004e-05 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2100 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7073843389999999, 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010216233999999999, 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.336 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2100 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525562+0000", 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3030000000000001e-06 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525569+0000", 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9240000000000001e-06 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525596+0000", 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7430999999999999e-05 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525645+0000", 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8597000000000002e-05 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535663+0000", 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010018132000000001 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535711+0000", 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7706e-05 2024-09-16T11:13:30.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535777+0000", 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6141000000000003e-05 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2101 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7072848760000001, 2024-09-16T11:13:30.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010279129999999999, 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2101 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525662+0000", 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.437e-06 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525668+0000", 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0900000000000001e-06 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525709+0000", 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1464000000000001e-05 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525726+0000", 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7210999999999998e-05 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535807+0000", 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010080621999999999 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535872+0000", 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5504000000000002e-05 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535939+0000", 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6802000000000002e-05 2024-09-16T11:13:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2266 4.8 4:11824548:::rbd_data.11b5220ac4a2.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.538416+0000", 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.6945293679999995, 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070944470000000003, 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2266 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.538416+0000", 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.538416+0000", 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.538418+0000", 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.277e-06 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539392+0000", 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00097422399999999999 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539413+0000", 2024-09-16T11:13:30.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0610000000000001e-05 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539421+0000", 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0139999999999998e-06 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539482+0000", 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1396999999999995e-05 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539606+0000", 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012412400000000001 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539817+0000", 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000210745 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.543132+0000", 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033147290000000002 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.545419+0000", 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022871559999999998 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.545470+0000", 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1020000000000003e-05 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.545510+0000", 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0151000000000002e-05 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.350 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2720 4.15 4:a82fda07:::rbd_data.11b5220ac4a2.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:25.388510+0000", 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.8444349249999998, 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075686929999999996, 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.351 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2720 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388510+0000", 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388510+0000", 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388511+0000", 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388515+0000", 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5370000000000002e-06 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388534+0000", 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9040000000000001e-05 2024-09-16T11:13:30.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388540+0000", 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6499999999999999e-06 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.389992+0000", 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001451379 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.390446+0000", 2024-09-16T11:13:30.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045423700000000002 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.390831+0000", 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038509200000000002 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.393896+0000", 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003064893 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.395936+0000", 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020395560000000001 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396020+0000", 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4237e-05 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396079+0000", 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8998e-05 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.356 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2723 4.15 4:ac84a83f:::rbd_data.11b5220ac4a2.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:25.388715+0000", 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.8442305120000002, 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007568155, 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2723 2024-09-16T11:13:30.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388715+0000", 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388715+0000", 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388716+0000", 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.232e-06 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388722+0000", 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4230000000000002e-06 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388739+0000", 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-05 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388762+0000", 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3482999999999999e-05 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.391820+0000", 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030575260000000001 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.392042+0000", 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022243999999999999 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.392205+0000", 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016326900000000001 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.395502+0000", 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032966580000000001 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396159+0000", 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00065711999999999997 2024-09-16T11:13:30.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396229+0000", 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0193999999999999e-05 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396283+0000", 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.321e-05 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.362 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.363 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:30.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:30.365 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:30.361+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:30.434 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T11:13:30.465 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:30.461+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:30.466 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:30.466 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:30.466 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:30.466 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:30.476 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:30.513 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:30.566 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:30.561+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:30.572 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:30.573 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:30.573 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:30.573 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:30.573 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:30.582 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:30.653 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:30.667 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:30.661+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:30.674 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:30.682 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.547786561000001, 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066095169999999996, 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 11 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:30.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130364+0000", 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2200000000000001e-06 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130378+0000", 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3498e-05 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130415+0000", 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6928999999999998e-05 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130431+0000", 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5834000000000001e-05 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130591+0000", 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016076400000000001 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130610+0000", 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8244999999999999e-05 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.133935+0000", 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033251650000000002 2024-09-16T11:13:30.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134174+0000", 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023965400000000001 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134553+0000", 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037902299999999998 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136098+0000", 2024-09-16T11:13:30.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0015447010000000001 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136814+0000", 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00071608299999999995 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136888+0000", 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3526e-05 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136971+0000", 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2874999999999995e-05 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:544 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.380772053999999, 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073024969999999998, 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.689 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 544 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:30.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297378+0000", 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.99e-06 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297388+0000", 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0407e-05 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297415+0000", 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7498e-05 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297426+0000", 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0957e-05 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297594+0000", 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167157 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297849+0000", 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025576 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.298833+0000", 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098353599999999992 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300723+0000", 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001890395 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304585+0000", 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038620709999999999 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304639+0000", 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3193999999999999e-05 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304678+0000", 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9532e-05 2024-09-16T11:13:30.694 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:549 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.380343679999999, 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007009576, 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 549 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297806+0000", 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.747e-06 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297814+0000", 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2900000000000002e-06 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297858+0000", 2024-09-16T11:13:30.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4295000000000002e-05 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297868+0000", 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2639999999999997e-06 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299859+0000", 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019912609999999998 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299944+0000", 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5069999999999997e-05 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300312+0000", 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036818899999999999 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304727+0000", 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044150689999999998 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304734+0000", 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304779+0000", 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5055000000000001e-05 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304814+0000", 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4770000000000001e-05 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.700 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:591 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.377549891999999, 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043393090000000004, 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 591 2024-09-16T11:13:30.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300600+0000", 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1320000000000001e-06 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300611+0000", 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0961000000000001e-05 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300633+0000", 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1999000000000001e-05 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300644+0000", 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1632e-05 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300770+0000", 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012577899999999999 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300927+0000", 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000157186 2024-09-16T11:13:30.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301666+0000", 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073808200000000004 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304821+0000", 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031557349999999998 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304859+0000", 2024-09-16T11:13:30.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7450999999999998e-05 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304902+0000", 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3579999999999999e-05 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304937+0000", 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4771999999999998e-05 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:592 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.377269052000001, 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061751929999999998, 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 592 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.707 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300880+0000", 2024-09-16T11:13:30.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6080000000000001e-06 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300890+0000", 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4040000000000001e-06 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300907+0000", 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6903000000000001e-05 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300916+0000", 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5219999999999996e-06 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302514+0000", 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00159807 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302687+0000", 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000173155 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303339+0000", 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0006514 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306894+0000", 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035556720000000002 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306940+0000", 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5893999999999998e-05 2024-09-16T11:13:30.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307005+0000", 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4432999999999998e-05 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307054+0000", 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9132000000000003e-05 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:593 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.376963898, 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082575289999999996, 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 593 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301185+0000", 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2860000000000001e-06 2024-09-16T11:13:30.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301193+0000", 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.678e-06 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301209+0000", 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5797000000000001e-05 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301218+0000", 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1980000000000007e-06 2024-09-16T11:13:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303958+0000", 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027405810000000002 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304015+0000", 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7074000000000003e-05 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304249+0000", 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023344199999999999 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307114+0000", 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028655569999999999 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309341+0000", 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022266339999999999 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309401+0000", 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0436000000000002e-05 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309441+0000", 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9845999999999998e-05 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:595 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:30.718 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.376912982, 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010351252, 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 595 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2999999999999999e-07 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301240+0000", 2024-09-16T11:13:30.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8359999999999999e-06 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301269+0000", 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8459e-05 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301278+0000", 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7949999999999996e-06 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304969+0000", 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003691864 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305136+0000", 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016632999999999999 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305576+0000", 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043990000000000001 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309262+0000", 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036865930000000002 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311524+0000", 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022620909999999999 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311559+0000", 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4597000000000001e-05 2024-09-16T11:13:30.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311586+0000", 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6957e-05 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:597 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.376770999, 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010265962, 2024-09-16T11:13:30.724 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 597 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301378+0000", 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.657e-06 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301387+0000", 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3860000000000007e-06 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301402+0000", 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4875000000000001e-05 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301411+0000", 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5820000000000008e-06 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305997+0000", 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045854169999999996 2024-09-16T11:13:30.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306078+0000", 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1614999999999997e-05 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306441+0000", 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036263000000000002 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309329+0000", 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028879650000000002 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311596+0000", 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022673939999999998 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311621+0000", 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4797e-05 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311643+0000", 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1644000000000001e-05 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:598 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.376687359, 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010230927000000001, 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 598 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301462+0000", 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301470+0000", 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6279999999999996e-06 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301484+0000", 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4314999999999999e-05 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301512+0000", 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8218999999999999e-05 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307142+0000", 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056298169999999996 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307264+0000", 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121704 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307701+0000", 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043725099999999999 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311457+0000", 2024-09-16T11:13:30.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037559 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311649+0000", 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019221900000000001 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311673+0000", 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3603000000000001e-05 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311691+0000", 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8609000000000001e-05 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:599 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.376476093000001, 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014841653999999999, 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 599 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301673+0000", 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3459999999999999e-06 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301681+0000", 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0949999999999996e-06 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301696+0000", 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4932e-05 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301705+0000", 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6009999999999995e-06 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308183+0000", 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064783840000000002 2024-09-16T11:13:30.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308260+0000", 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6601999999999996e-05 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308708+0000", 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044877600000000001 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311516+0000", 2024-09-16T11:13:30.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028079189999999999 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316392+0000", 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004875735 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316467+0000", 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4571000000000001e-05 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316513+0000", 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6693000000000002e-05 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:601 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.376009797, 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014436584000000001, 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 601 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:30.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302139+0000", 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302152+0000", 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3608000000000001e-05 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302163+0000", 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1012000000000001e-05 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302169+0000", 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1309999999999997e-06 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309509+0000", 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073404869999999997 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309603+0000", 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4374000000000001e-05 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310124+0000", 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052012300000000005 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312552+0000", 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024287969999999999 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316527+0000", 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039741569999999999 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316553+0000", 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945999999999999e-05 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316574+0000", 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1937999999999999e-05 2024-09-16T11:13:30.747 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:603 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.375324357, 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019179324000000001, 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.748 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 603 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302824+0000", 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.06e-06 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302830+0000", 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9209999999999997e-06 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302844+0000", 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3304e-05 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302850+0000", 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310529+0000", 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076792980000000002 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310601+0000", 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1070999999999998e-05 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311059+0000", 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045884099999999998 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313207+0000", 2024-09-16T11:13:30.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002147805 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321862+0000", 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086545330000000007 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321919+0000", 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7130999999999999e-05 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322003+0000", 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3794000000000004e-05 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:604 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.374518361, 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.018458927999999999, 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 604 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:30.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303631+0000", 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5579999999999999e-06 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303638+0000", 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303654+0000", 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6628000000000001e-05 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303662+0000", 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8390000000000007e-06 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311727+0000", 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080650300000000008 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311786+0000", 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8507000000000001e-05 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312144+0000", 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358821 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.314152+0000", 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002008045 2024-09-16T11:13:30.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322013+0000", 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078606920000000007 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322053+0000", 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322088+0000", 2024-09-16T11:13:30.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4972000000000003e-05 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:605 4.13 e38/31 4:cae7a2e7:::rbd_data.11b52605a025.0000000000000001:head v 38'7, mlcod=35'4)", 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.372586494, 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056889410000000003, 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305562+0000", 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:30.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305567+0000", 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5949999999999998e-06 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305577+0000", 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305584+0000", 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3670000000000002e-06 2024-09-16T11:13:30.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305659+0000", 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5593999999999995e-05 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305672+0000", 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3193000000000001e-05 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311201+0000", 2024-09-16T11:13:30.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055283249999999997 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311250+0000", 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9382000000000003e-05 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:613 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'8, mlcod=37'5)", 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.365618975, 2024-09-16T11:13:30.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095140820000000001, 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312549+0000", 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9432999999999998e-05 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312559+0000", 2024-09-16T11:13:30.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0455000000000001e-05 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312565+0000", 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4609999999999999e-06 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312594+0000", 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9456000000000001e-05 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312622+0000", 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7586999999999999e-05 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322004+0000", 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093818349999999998 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322043+0000", 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9167e-05 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:614 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'10, mlcod=37'5)", 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.365127963999999, 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010261963000000001, 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.769 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:30.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313021+0000", 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5799999999999998e-07 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313028+0000", 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1400000000000002e-06 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313041+0000", 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2958e-05 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313047+0000", 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0530000000000003e-06 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313090+0000", 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2936999999999999e-05 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313105+0000", 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5199e-05 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323210+0000", 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010104958000000001 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323282+0000", 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1860000000000007e-05 2024-09-16T11:13:30.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:617 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.361367828000001, 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060101130000000001, 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.774 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 617 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316781+0000", 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.279e-06 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316910+0000", 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012939700000000001 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316922+0000", 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1894e-05 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316928+0000", 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5559999999999998e-06 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316974+0000", 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5587000000000002e-05 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317094+0000", 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000120586 2024-09-16T11:13:30.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317242+0000", 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147517 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322697+0000", 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054555960000000001 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322708+0000", 2024-09-16T11:13:30.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0805000000000001e-05 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322745+0000", 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7280000000000002e-05 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322790+0000", 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4616000000000001e-05 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:618 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.361205972, 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006757379, 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 618 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.780 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:30.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316943+0000", 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9199999999999999e-07 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316945+0000", 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5009999999999999e-06 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316952+0000", 2024-09-16T11:13:30.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5980000000000002e-06 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316955+0000", 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4879999999999999e-06 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318095+0000", 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011395050000000001 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318151+0000", 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6577999999999999e-05 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318284+0000", 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013229499999999999 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323278+0000", 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049947200000000002 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323605+0000", 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032718800000000001 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323658+0000", 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2961999999999999e-05 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323699+0000", 2024-09-16T11:13:30.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0652000000000003e-05 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2550 4.5 4:a5a2f6f0:::rbd_data.11b5220ac4a2.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.964379+0000", 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7137690430000001, 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066632310000000004, 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.787 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2550 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964379+0000", 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964379+0000", 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964382+0000", 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.41e-06 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964408+0000", 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6284000000000001e-05 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964464+0000", 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5875999999999998e-05 2024-09-16T11:13:30.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964488+0000", 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4117e-05 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964668+0000", 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017928900000000001 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964890+0000", 2024-09-16T11:13:30.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022197699999999999 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.965202+0000", 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031292700000000001 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.966789+0000", 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001586495 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.970599+0000", 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038096250000000001 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.970920+0000", 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320907 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.971042+0000", 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012232399999999999 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.792 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2557 4.a 4:5121c3d4:::rbd_data.11b5220ac4a2.0000000000000043:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.967302+0000", 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.7108452820000002, 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039395630000000001, 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2557 2024-09-16T11:13:30.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967302+0000", 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967302+0000", 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967305+0000", 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2759999999999999e-06 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967318+0000", 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2984e-05 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967348+0000", 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0031000000000001e-05 2024-09-16T11:13:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967363+0000", 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5247e-05 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967521+0000", 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015815499999999999 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.967743+0000", 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022212100000000001 2024-09-16T11:13:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.968483+0000", 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073994399999999998 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.970254+0000", 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0017707199999999999 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.971007+0000", 2024-09-16T11:13:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00075306099999999996 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.971114+0000", 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107447 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.971242+0000", 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000127577 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:30.799 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:30.799 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:30.800 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:30.800 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:30.800 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:30.800 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:30.801 INFO:teuthology.orchestra.run.smithi162.stdout:845 read 0x107b3d2 thru 0x1085dfc (0xaa2b bytes) 2024-09-16T11:13:30.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:30.802 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:30.802 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:30.793+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:30.808 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:30.811 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:30.811 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:30.811 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:30.811 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:30.811 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:30.812 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:30.812 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:30.812 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:30.822 INFO:teuthology.orchestra.run.smithi162.stdout:846 write 0x2ef353b thru 0x2f01d88 (0xe84e bytes) 2024-09-16T11:13:30.901 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:30.897+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:30.917 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:30.917 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:30.920 INFO:teuthology.orchestra.run.smithi162.stdout:848 read 0x12b93af thru 0x12c8f07 (0xfb59 bytes) 2024-09-16T11:13:30.926 INFO:teuthology.orchestra.run.smithi162.stdout:849 punch from 0x26efbf to 0x27722a, (0x826b bytes) 2024-09-16T11:13:30.926 INFO:teuthology.orchestra.run.smithi162.stdout:850 read 0x13fac88 thru 0x140372d (0x8aa6 bytes) 2024-09-16T11:13:30.928 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:30.942 INFO:teuthology.orchestra.run.smithi162.stdout:851 punch from 0x17c258f to 0x17c8688, (0x60f9 bytes) 2024-09-16T11:13:30.957 INFO:teuthology.orchestra.run.smithi162.stdout:852 writesame 0x95b885f thru 0x95bfe3c data_size 0x2f(0x75de bytes) 2024-09-16T11:13:30.957 INFO:teuthology.orchestra.run.smithi162.stdout:853 punch from 0x2127ff0 to 0x2128acb, (0xadb bytes) 2024-09-16T11:13:30.957 INFO:teuthology.orchestra.run.smithi162.stdout:854 read 0x2d11d8f thru 0x2d13889 (0x1afb bytes) 2024-09-16T11:13:30.958 INFO:teuthology.orchestra.run.smithi162.stdout:856 read 0x1cc1ce1 thru 0x1cc35fe (0x191e bytes) 2024-09-16T11:13:30.959 INFO:teuthology.orchestra.run.smithi162.stdout:857 punch from 0x72a4cef to 0x72a7859, (0x2b6a bytes) 2024-09-16T11:13:30.959 INFO:teuthology.orchestra.run.smithi162.stdout:860 write 0x4d80be2 thru 0x4d8a7a0 (0x9bbf bytes) 2024-09-16T11:13:31.002 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:30.997+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:31.038 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.039 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.040 INFO:teuthology.orchestra.run.smithi162.stdout:861 read 0x4f6be2 thru 0x4fbebc (0x52db bytes) 2024-09-16T11:13:31.050 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:31.072 INFO:teuthology.orchestra.run.smithi162.stdout:863 read 0x3d6d61b thru 0x3d6f7c7 (0x21ad bytes) 2024-09-16T11:13:31.082 INFO:teuthology.orchestra.run.smithi162.stdout:864 read 0x5829da5 thru 0x583363d (0x9899 bytes) 2024-09-16T11:13:31.104 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:31.097+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:31.130 INFO:teuthology.orchestra.run.smithi162.stdout:865 write 0xd5b3906 thru 0xd5c1448 (0xdb43 bytes) 2024-09-16T11:13:31.156 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.156 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.163 INFO:teuthology.orchestra.run.smithi162.stdout:866 writesame 0xd78dfef thru 0xd79db99 data_size 0x1(0xfbab bytes) 2024-09-16T11:13:31.166 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:31.178 INFO:teuthology.orchestra.run.smithi162.stdout:867 write 0xdca2eef thru 0xdca7550 (0x4662 bytes) 2024-09-16T11:13:31.183 INFO:teuthology.orchestra.run.smithi162.stdout:868 write 0xb8e6c2c thru 0xb8f0937 (0x9d0c bytes) 2024-09-16T11:13:31.204 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:31.201+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:31.211 INFO:teuthology.orchestra.run.smithi162.stdout:869 trunc from 0xdca7551 to 0x94499a5 2024-09-16T11:13:31.229 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:13:31.240 INFO:teuthology.orchestra.run.smithi162.stdout:871 read 0x8275819 thru 0x8279bc4 (0x43ac bytes) 2024-09-16T11:13:31.240 INFO:teuthology.orchestra.run.smithi162.stdout:872 punch from 0x49b1aa8 to 0x49bb785, (0x9cdd bytes) 2024-09-16T11:13:31.240 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0xeaeac2e 2024-09-16T11:13:31.248 INFO:teuthology.orchestra.run.smithi162.stdout:873 trunc from 0x94499a5 to 0xeaeac2e 2024-09-16T11:13:31.257 INFO:teuthology.orchestra.run.smithi162.stdout:874 write 0xb301776 thru 0xb30ef21 (0xd7ac bytes) 2024-09-16T11:13:31.282 INFO:teuthology.orchestra.run.smithi162.stdout:875 write 0xd90fa57 thru 0xd91f6ce (0xfc78 bytes) 2024-09-16T11:13:31.288 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.288 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.299 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:31.304 INFO:teuthology.orchestra.run.smithi162.stdout:876 punch from 0x5528ad4 to 0x552cd33, (0x425f bytes) 2024-09-16T11:13:31.305 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:31.301+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:31.320 INFO:teuthology.orchestra.run.smithi162.stdout:877 trunc from 0xeaeac2e to 0x3ba2c1f 2024-09-16T11:13:31.406 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:31.401+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:31.420 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.420 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:31.444 INFO:teuthology.orchestra.run.smithi162.stdout:878 read 0x1d97d5a thru 0x1da6245 (0xe4ec bytes) 2024-09-16T11:13:31.448 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:31.448 INFO:teuthology.orchestra.run.smithi162.stdout:880 read 0x3d3c83 thru 0x3d3db9 (0x137 bytes) 2024-09-16T11:13:31.451 INFO:teuthology.orchestra.run.smithi162.stdout:881 read 0x1aecb8d thru 0x1af94cf (0xc943 bytes) 2024-09-16T11:13:31.454 INFO:teuthology.orchestra.run.smithi162.stdout:882 punch from 0x298ea5a to 0x299d1ba, (0xe760 bytes) 2024-09-16T11:13:31.454 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:31.477 INFO:teuthology.orchestra.run.smithi162.stdout:884 write 0x83d4dd0 thru 0x83d6bf2 (0x1e23 bytes) 2024-09-16T11:13:31.484 INFO:teuthology.orchestra.run.smithi162.stdout:886 punch from 0x4c78f1e to 0x4c84b10, (0xbbf2 bytes) 2024-09-16T11:13:31.484 INFO:teuthology.orchestra.run.smithi162.stdout:888 read 0xa375af thru 0xa3b496 (0x3ee8 bytes) 2024-09-16T11:13:31.484 INFO:teuthology.orchestra.run.smithi162.stdout:889 writesame 0x2f2f72d thru 0x2f33b24 data_size 0x57(0x43f8 bytes) 2024-09-16T11:13:31.489 INFO:teuthology.orchestra.run.smithi162.stdout:890 read 0x69bc41b thru 0x69c3946 (0x752c bytes) 2024-09-16T11:13:31.494 INFO:teuthology.orchestra.run.smithi162.stdout:894 trunc from 0x83d6bf3 to 0x4fd6547 2024-09-16T11:13:31.507 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:31.501+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:31.515 INFO:teuthology.orchestra.run.smithi162.stdout:895 writesame 0x1bb6b79 thru 0x1bba044 data_size 0x3e(0x34cc bytes) 2024-09-16T11:13:31.521 INFO:teuthology.orchestra.run.smithi162.stdout:896 read 0x132ee49 thru 0x1334700 (0x58b8 bytes) 2024-09-16T11:13:31.523 INFO:teuthology.orchestra.run.smithi162.stdout:897 read 0x33724e6 thru 0x3378c33 (0x674e bytes) 2024-09-16T11:13:31.547 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.547 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.552 INFO:teuthology.orchestra.run.smithi162.stdout:899 writesame 0xc7b15fd thru 0xc7b4dfe data_size 0x2(0x3802 bytes) 2024-09-16T11:13:31.552 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:31.552 INFO:teuthology.orchestra.run.smithi162.stdout:901 write 0x41b0f95 thru 0x41bbf2e (0xaf9a bytes) 2024-09-16T11:13:31.556 INFO:teuthology.orchestra.run.smithi162.stdout:903 write 0x87fd9f5 thru 0x8802328 (0x4934 bytes) 2024-09-16T11:13:31.558 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:31.571 INFO:teuthology.orchestra.run.smithi162.stdout:905 write 0x5b49878 thru 0x5b55a8f (0xc218 bytes) 2024-09-16T11:13:31.608 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:31.605+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:31.611 INFO:teuthology.orchestra.run.smithi162.stdout:906 write 0xe651252 thru 0xe652b28 (0x18d7 bytes) 2024-09-16T11:13:31.620 INFO:teuthology.orchestra.run.smithi162.stdout:907 write 0xc9fddc thru 0xca7634 (0x7859 bytes) 2024-09-16T11:13:31.685 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.685 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.689 INFO:teuthology.orchestra.run.smithi162.stdout:908 punch from 0x3b307b9 to 0x3b38dd9, (0x8620 bytes) 2024-09-16T11:13:31.689 INFO:teuthology.orchestra.run.smithi162.stdout:910 write 0x876d216 thru 0x8776588 (0x9373 bytes) 2024-09-16T11:13:31.695 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:31.701 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:31.722 INFO:teuthology.orchestra.run.smithi162.stdout:914 trunc from 0xe652b29 to 0x150b9be 2024-09-16T11:13:31.722 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:31.717+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:31.793 INFO:teuthology.orchestra.run.smithi162.stdout:916 write 0x9e8b8e2 thru 0x9e99941 (0xe060 bytes) 2024-09-16T11:13:31.808 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.808 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.809 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:31.805+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:31.820 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:31.829 INFO:teuthology.orchestra.run.smithi162.stdout:917 write 0xca622ea thru 0xca66953 (0x466a bytes) 2024-09-16T11:13:31.851 INFO:teuthology.orchestra.run.smithi162.stdout:918 writesame 0xcee1faf thru 0xcee3eda data_size 0x1e(0x1f2c bytes) 2024-09-16T11:13:31.853 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:31.853 INFO:teuthology.orchestra.run.smithi162.stdout:920 punch from 0x9da0c35 to 0x9da4a12, (0x3ddd bytes) 2024-09-16T11:13:31.853 INFO:teuthology.orchestra.run.smithi162.stdout:921 read 0xa471f2d thru 0xa478eab (0x6f7f bytes) 2024-09-16T11:13:31.854 INFO:teuthology.orchestra.run.smithi162.stdout:923 write 0x384658a thru 0x38550d6 (0xeb4d bytes) 2024-09-16T11:13:31.894 INFO:teuthology.orchestra.run.smithi162.stdout:924 trunc from 0xcee3edb to 0x15cbfd9 2024-09-16T11:13:31.910 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:31.905+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:31.942 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:31.943 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:31.946 INFO:teuthology.orchestra.run.smithi162.stdout:925 write 0xc3f25a4 thru 0xc3f7f86 (0x59e3 bytes) 2024-09-16T11:13:31.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:31.960 INFO:teuthology.orchestra.run.smithi162.stdout:926 read 0x9a2eb33 thru 0x9a36804 (0x7cd2 bytes) 2024-09-16T11:13:31.960 INFO:teuthology.orchestra.run.smithi162.stdout:927 punch from 0x154f2bd to 0x1551039, (0x1d7c bytes) 2024-09-16T11:13:31.960 INFO:teuthology.orchestra.run.smithi162.stdout:929 punch from 0x1bc3ab1 to 0x1bcbb8e, (0x80dd bytes) 2024-09-16T11:13:31.960 INFO:teuthology.orchestra.run.smithi162.stdout:931 write 0x3126679 thru 0x31287e3 (0x216b bytes) 2024-09-16T11:13:31.968 INFO:teuthology.orchestra.run.smithi162.stdout:932 writesame 0x245d649 thru 0x246559e data_size 0x9(0x7f56 bytes) 2024-09-16T11:13:31.985 INFO:teuthology.orchestra.run.smithi162.stdout:933 punch from 0xa13228b to 0xa132960, (0x6d5 bytes) 2024-09-16T11:13:31.985 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:31.985 INFO:teuthology.orchestra.run.smithi162.stdout:935 writesame 0x8fd9943 thru 0x8fe2adf data_size 0x1(0x919d bytes) 2024-09-16T11:13:32.007 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.007 INFO:teuthology.orchestra.run.smithi162.stdout:937 punch from 0x851764e to 0x85195b3, (0x1f65 bytes) 2024-09-16T11:13:32.010 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:32.005+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:32.016 INFO:teuthology.orchestra.run.smithi162.stdout:938 write 0xd66f2dd thru 0xd671148 (0x1e6c bytes) 2024-09-16T11:13:32.026 INFO:teuthology.orchestra.run.smithi162.stdout:939 write 0x92857d3 thru 0x928d058 (0x7886 bytes) 2024-09-16T11:13:32.035 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.035 INFO:teuthology.orchestra.run.smithi162.stdout:943 writesame 0x1216ee4 thru 0x1222b6d data_size 0x1(0xbc8a bytes) 2024-09-16T11:13:32.048 INFO:teuthology.orchestra.run.smithi162.stdout:944 writesame 0x4abc98f thru 0x4ac2af5 data_size 0x1(0x6167 bytes) 2024-09-16T11:13:32.060 INFO:teuthology.orchestra.run.smithi162.stdout:946 read 0x99ee82f thru 0x99f19d2 (0x31a4 bytes) 2024-09-16T11:13:32.061 INFO:teuthology.orchestra.run.smithi162.stdout:947 writesame 0x8596547 thru 0x85a2d5e data_size 0x4c(0xc818 bytes) 2024-09-16T11:13:32.074 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.074 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.080 INFO:teuthology.orchestra.run.smithi162.stdout:948 read 0xacc8692 thru 0xacca041 (0x19b0 bytes) 2024-09-16T11:13:32.080 INFO:teuthology.orchestra.run.smithi162.stdout:949 punch from 0x52db03 to 0x53d47c, (0xf979 bytes) 2024-09-16T11:13:32.080 INFO:teuthology.orchestra.run.smithi162.stdout:950 read 0x8724a46 thru 0x87309bb (0xbf76 bytes) 2024-09-16T11:13:32.081 INFO:teuthology.orchestra.run.smithi162.stdout:951 write 0x787cb9a thru 0x787f6db (0x2b42 bytes) 2024-09-16T11:13:32.085 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:32.095 INFO:teuthology.orchestra.run.smithi162.stdout:952 trunc from 0xd671149 to 0xbfade6f 2024-09-16T11:13:32.112 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:32.109+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:32.133 INFO:teuthology.orchestra.run.smithi162.stdout:953 trunc from 0xbfade6f to 0x4e4b37f 2024-09-16T11:13:32.174 INFO:teuthology.orchestra.run.smithi162.stdout:954 trunc from 0x4e4b37f to 0xe814f80 2024-09-16T11:13:32.184 INFO:teuthology.orchestra.run.smithi162.stdout:955 punch from 0x92274a9 to 0x922e9b3, (0x750a bytes) 2024-09-16T11:13:32.184 INFO:teuthology.orchestra.run.smithi162.stdout:956 read 0x2e6717e thru 0x2e6aa77 (0x38fa bytes) 2024-09-16T11:13:32.184 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.185 INFO:teuthology.orchestra.run.smithi162.stdout:958 writesame 0xa700869 thru 0xa703fd5 data_size 0x1(0x376d bytes) 2024-09-16T11:13:32.195 INFO:teuthology.orchestra.run.smithi162.stdout:959 punch from 0x483daeb to 0x4846c20, (0x9135 bytes) 2024-09-16T11:13:32.195 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.196 INFO:teuthology.orchestra.run.smithi162.stdout:964 read 0x7f56e44 thru 0x7f6594c (0xeb09 bytes) 2024-09-16T11:13:32.196 INFO:teuthology.orchestra.run.smithi162.stdout:965 trunc from 0xe814f80 to 0xe0b7a45 2024-09-16T11:13:32.206 INFO:teuthology.orchestra.run.smithi162.stdout:968 writesame 0x4789ca2 thru 0x478d196 data_size 0x1(0x34f5 bytes) 2024-09-16T11:13:32.207 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.208 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.213 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:32.209+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:32.220 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:32.222 INFO:teuthology.orchestra.run.smithi162.stdout:969 read 0x625de44 thru 0x626d537 (0xf6f4 bytes) 2024-09-16T11:13:32.243 INFO:teuthology.orchestra.run.smithi162.stdout:970 trunc from 0xe0b7a45 to 0x5220ac 2024-09-16T11:13:32.314 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:32.309+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:32.326 INFO:teuthology.orchestra.run.smithi162.stdout:972 read 0x49f5f9 thru 0x4a26fb (0x3103 bytes) 2024-09-16T11:13:32.327 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.327 INFO:teuthology.orchestra.run.smithi162.stdout:974 punch from 0x26fae6 to 0x27776f, (0x7c89 bytes) 2024-09-16T11:13:32.327 INFO:teuthology.orchestra.run.smithi162.stdout:975 read 0x35e5bb thru 0x35fea6 (0x18ec bytes) 2024-09-16T11:13:32.344 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.344 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.355 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:32.357 INFO:teuthology.orchestra.run.smithi162.stdout:976 writesame 0xb009090 thru 0xb012dbd data_size 0x3e(0x9d2e bytes) 2024-09-16T11:13:32.357 INFO:teuthology.orchestra.run.smithi162.stdout:977 read 0x9d1616d thru 0x9d16edd (0xd71 bytes) 2024-09-16T11:13:32.357 INFO:teuthology.orchestra.run.smithi162.stdout:978 write 0x7305a08 thru 0x730b0f5 (0x56ee bytes) 2024-09-16T11:13:32.381 INFO:teuthology.orchestra.run.smithi162.stdout:979 punch from 0x944d0a6 to 0x945b606, (0xe560 bytes) 2024-09-16T11:13:32.381 INFO:teuthology.orchestra.run.smithi162.stdout:981 read 0x7e39a5c thru 0x7e3f3de (0x5983 bytes) 2024-09-16T11:13:32.381 INFO:teuthology.orchestra.run.smithi162.stdout:982 read 0xaf3f0d0 thru 0xaf46a68 (0x7999 bytes) 2024-09-16T11:13:32.381 INFO:teuthology.orchestra.run.smithi162.stdout:984 writesame 0x74f8fc thru 0x75dedd data_size 0x6e(0xe5e2 bytes) 2024-09-16T11:13:32.387 INFO:teuthology.orchestra.run.smithi162.stdout:987 write 0x7036022 thru 0x703621b (0x1fa bytes) 2024-09-16T11:13:32.391 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.392 INFO:teuthology.orchestra.run.smithi162.stdout:989 punch from 0x7502d96 to 0x75034bf, (0x729 bytes) 2024-09-16T11:13:32.399 INFO:teuthology.orchestra.run.smithi162.stdout:990 trunc from 0xb012dbe to 0x6147392 2024-09-16T11:13:32.416 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:32.409+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:32.439 INFO:teuthology.orchestra.run.smithi162.stdout:991 write 0xb6ea925 thru 0xb6f973d (0xee19 bytes) 2024-09-16T11:13:32.465 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.465 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.468 INFO:teuthology.orchestra.run.smithi162.stdout:992 read 0x345a7a thru 0x355a63 (0xffea bytes) 2024-09-16T11:13:32.469 INFO:teuthology.orchestra.run.smithi162.stdout:993 writesame 0xb0b66ac thru 0xb0c1eae data_size 0x11(0xb803 bytes) 2024-09-16T11:13:32.476 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:32.491 INFO:teuthology.orchestra.run.smithi162.stdout:994 punch from 0x4a8add1 to 0x4a8f645, (0x4874 bytes) 2024-09-16T11:13:32.491 INFO:teuthology.orchestra.run.smithi162.stdout:995 write 0x85981ab thru 0x85a6b42 (0xe998 bytes) 2024-09-16T11:13:32.518 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.518 INFO:teuthology.orchestra.run.smithi162.stdout:997 write 0x6edb859 thru 0x6ee36b0 (0x7e58 bytes) 2024-09-16T11:13:32.518 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:32.513+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:32.539 INFO:teuthology.orchestra.run.smithi162.stdout:998 trunc from 0xb6f973e to 0x842ac6c 2024-09-16T11:13:32.564 INFO:teuthology.orchestra.run.smithi162.stdout:999 read 0x7d07fc0 thru 0x7d15275 (0xd2b6 bytes) 2024-09-16T11:13:32.564 INFO:teuthology.orchestra.run.smithi162.stdout:1000 read 0x7ed3d68 thru 0x7ed607b (0x2314 bytes) 2024-09-16T11:13:32.565 INFO:teuthology.orchestra.run.smithi162.stdout:1001 read 0x17ad137 thru 0x17b1b67 (0x4a31 bytes) 2024-09-16T11:13:32.572 INFO:teuthology.orchestra.run.smithi162.stdout:1002 trunc from 0x842ac6c to 0xcd14f16 2024-09-16T11:13:32.586 INFO:teuthology.orchestra.run.smithi162.stdout:1003 trunc from 0xcd14f16 to 0x9bc12f6 2024-09-16T11:13:32.596 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.596 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.597 INFO:teuthology.orchestra.run.smithi162.stdout:1004 read 0x866195b thru 0x866d7d7 (0xbe7d bytes) 2024-09-16T11:13:32.598 INFO:teuthology.orchestra.run.smithi162.stdout:1006 read 0x7833b3d thru 0x783ebcb (0xb08f bytes) 2024-09-16T11:13:32.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:32.615 INFO:teuthology.orchestra.run.smithi162.stdout:1008 writesame 0xbcc62dd thru 0xbcd364f data_size 0x85(0xd373 bytes) 2024-09-16T11:13:32.615 INFO:teuthology.orchestra.run.smithi162.stdout:1009 read 0x54eb4b7 thru 0x54ec42a (0xf74 bytes) 2024-09-16T11:13:32.615 INFO:teuthology.orchestra.run.smithi162.stdout:1011 write 0x5ba7810 thru 0x5bad9c0 (0x61b1 bytes) 2024-09-16T11:13:32.617 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:32.613+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:32.636 INFO:teuthology.orchestra.run.smithi162.stdout:1012 write 0xab3ad66 thru 0xab46dcb (0xc066 bytes) 2024-09-16T11:13:32.661 INFO:teuthology.orchestra.run.smithi162.stdout:1014 writesame 0x15392e3 thru 0x153dce2 data_size 0x25(0x4a00 bytes) 2024-09-16T11:13:32.688 INFO:teuthology.orchestra.run.smithi162.stdout:1015 trunc from 0xbcd3650 to 0x28f0565 2024-09-16T11:13:32.713 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.713 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.713 INFO:teuthology.orchestra.run.smithi162.stdout:1018 read 0xa167f4 thru 0xa260b5 (0xf8c2 bytes) 2024-09-16T11:13:32.718 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:32.713+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:32.724 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.725 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.730 INFO:teuthology.orchestra.run.smithi162.stdout:1020 write 0x6d1c333 thru 0x6d22c55 (0x6923 bytes) 2024-09-16T11:13:32.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:32.747 INFO:teuthology.orchestra.run.smithi162.stdout:1023 read 0x4b26cf6 thru 0x4b2af3e (0x4249 bytes) 2024-09-16T11:13:32.747 INFO:teuthology.orchestra.run.smithi162.stdout:1025 punch from 0xe6dcc8 to 0xe7b791, (0xdac9 bytes) 2024-09-16T11:13:32.753 INFO:teuthology.orchestra.run.smithi162.stdout:1026 trunc from 0x6d22c56 to 0x9fb5be6 2024-09-16T11:13:32.764 INFO:teuthology.orchestra.run.smithi162.stdout:1027 writesame 0x5c20d0f thru 0x5c24899 data_size 0x1(0x3b8b bytes) 2024-09-16T11:13:32.782 INFO:teuthology.orchestra.run.smithi162.stdout:1028 write 0x9cada22 thru 0x9cbc971 (0xef50 bytes) 2024-09-16T11:13:32.810 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.810 INFO:teuthology.orchestra.run.smithi162.stdout:1031 read 0x834f647 thru 0x83571c9 (0x7b83 bytes) 2024-09-16T11:13:32.811 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.811 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0xedc02da 2024-09-16T11:13:32.819 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:32.817+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:32.819 INFO:teuthology.orchestra.run.smithi162.stdout:1034 trunc from 0x9fb5be6 to 0xedc02da 2024-09-16T11:13:32.826 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:32.830 INFO:teuthology.orchestra.run.smithi162.stdout:1035 write 0x4e932e0 thru 0x4e95885 (0x25a6 bytes) 2024-09-16T11:13:32.840 INFO:teuthology.orchestra.run.smithi162.stdout:1036 write 0x59d7ee0 thru 0x59de9e9 (0x6b0a bytes) 2024-09-16T11:13:32.851 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.851 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.855 INFO:teuthology.orchestra.run.smithi162.stdout:1037 write 0x6bba56f thru 0x6bc9cfa (0xf78c bytes) 2024-09-16T11:13:32.862 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:32.870 INFO:teuthology.orchestra.run.smithi162.stdout:1038 punch from 0x94bb508 to 0x94caa14, (0xf50c bytes) 2024-09-16T11:13:32.870 INFO:teuthology.orchestra.run.smithi162.stdout:1039 read 0x41f6809 thru 0x41fac8f (0x4487 bytes) 2024-09-16T11:13:32.871 INFO:teuthology.orchestra.run.smithi162.stdout:1040 write 0x7b5d9ca thru 0x7b6d98d (0xffc4 bytes) 2024-09-16T11:13:32.895 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.913 INFO:teuthology.orchestra.run.smithi162.stdout:1043 trunc from 0xedc02da to 0x26959c1 2024-09-16T11:13:32.920 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:32.913+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:32.960 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:32.961 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:32.961 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:32.961 INFO:teuthology.orchestra.run.smithi162.stdout:1045 write 0x8e24dac thru 0x8e30019 (0xb26e bytes) 2024-09-16T11:13:32.970 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:32.985 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:32.985 INFO:teuthology.orchestra.run.smithi162.stdout:1046 punch from 0x8d098a0 to 0x8d1383a, (0x9f9a bytes) 2024-09-16T11:13:32.986 INFO:teuthology.orchestra.run.smithi162.stdout:1047 read 0x87a271d thru 0x87a6d79 (0x465d bytes) 2024-09-16T11:13:32.986 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:32.986 INFO:teuthology.orchestra.run.smithi162.stdout:1049 read 0x56a13a thru 0x57029b (0x6162 bytes) 2024-09-16T11:13:32.992 INFO:teuthology.orchestra.run.smithi162.stdout:1050 trunc from 0x8e3001a to 0x5012362 2024-09-16T11:13:33.007 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:33.020 INFO:teuthology.orchestra.run.smithi162.stdout:1051 trunc from 0x5012362 to 0xaf669a9 2024-09-16T11:13:33.020 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:33.013+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:33.024 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:33.043 INFO:teuthology.orchestra.run.smithi162.stdout:1052 writesame 0xba8b784 thru 0xba94254 data_size 0x1(0x8ad1 bytes) 2024-09-16T11:13:33.043 INFO:teuthology.orchestra.run.smithi162.stdout:1053 write 0x63a66a2 thru 0x63b268e (0xbfed bytes) 2024-09-16T11:13:33.045 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:33.064 INFO:teuthology.orchestra.run.smithi162.stdout:1054 punch from 0x7caae3c to 0x7cb9881, (0xea45 bytes) 2024-09-16T11:13:33.064 INFO:teuthology.orchestra.run.smithi162.stdout:1055 writesame 0x6ef3fc3 thru 0x6efc010 data_size 0xb(0x804e bytes) 2024-09-16T11:13:33.071 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.071 INFO:teuthology.orchestra.run.smithi162.stdout:1058 write 0x397f80b thru 0x3989665 (0x9e5b bytes) 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:9 4.1a 4:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 17.958485739, 2024-09-16T11:13:33.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080919660000000008, 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 9 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108607+0000", 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108611+0000", 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4970000000000001e-06 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108629+0000", 2024-09-16T11:13:33.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8527999999999999e-05 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108666+0000", 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6965000000000003e-05 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108686+0000", 2024-09-16T11:13:33.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9457999999999999e-05 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108888+0000", 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020222999999999999 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.108902+0000", 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4511999999999999e-05 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.112496+0000", 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035939050000000001 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.112726+0000", 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000229559 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.113104+0000", 2024-09-16T11:13:33.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000377592 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.114998+0000", 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018945019999999999 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116382+0000", 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001384159 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116556+0000", 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000174289 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.116699+0000", 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014276999999999999 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:613 4.13 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:33.079 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.311926+0000", 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.755167467, 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010525013, 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 613 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311926+0000", 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311926+0000", 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311927+0000", 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.449e-06 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311953+0000", 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6435999999999999e-05 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311970+0000", 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7054e-05 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311990+0000", 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9157999999999999e-05 2024-09-16T11:13:33.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312056+0000", 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6246000000000002e-05 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312224+0000", 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016762600000000001 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312442+0000", 2024-09-16T11:13:33.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021867300000000001 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316530+0000", 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040882239999999997 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322252+0000", 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057218080000000001 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322398+0000", 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014619200000000001 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322451+0000", 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2147000000000001e-05 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:614 4.13 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head [truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:33.085 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.312014+0000", 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.755078829, 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011819527, 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 614 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312014+0000", 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312014+0000", 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312015+0000", 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8299999999999996e-07 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312018+0000", 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312028+0000", 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4830000000000005e-06 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312033+0000", 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5249999999999996e-06 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312760+0000", 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000726759 2024-09-16T11:13:33.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312820+0000", 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0334999999999997e-05 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312955+0000", 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000134411 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.319112+0000", 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061578800000000001 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323699+0000", 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045862710000000003 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323770+0000", 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0999000000000002e-05 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323834+0000", 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3930000000000006e-05 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:872 4.13 4:ca9f37af:::rbd_data.11b585795d6b.0000000000000003:head [stat,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.150021384, 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010169020000000001, 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.092 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 872 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:33.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917072+0000", 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7999999999999995e-07 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917075+0000", 2024-09-16T11:13:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.932e-06 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917085+0000", 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0248000000000001e-05 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.917091+0000", 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9569999999999999e-06 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.921551+0000", 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044594609999999996 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.921682+0000", 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013097 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.921849+0000", 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167348 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.922837+0000", 2024-09-16T11:13:33.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098752400000000004 2024-09-16T11:13:33.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.927113+0000", 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0042765019999999997 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.927183+0000", 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9698999999999998e-05 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.927241+0000", 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7799e-05 2024-09-16T11:13:33.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:891 4.9 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 8388608~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.138047696999999, 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015304566, 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 891 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929047+0000", 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4729999999999999e-06 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933394+0000", 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043469529999999998 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933425+0000", 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1238999999999997e-05 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933441+0000", 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5719999999999999e-05 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933522+0000", 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1339000000000002e-05 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933648+0000", 2024-09-16T11:13:33.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000125883 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933908+0000", 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026008699999999999 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.942070+0000", 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0081615249999999993 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944253+0000", 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021831910000000001 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944310+0000", 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7241999999999998e-05 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944350+0000", 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9913999999999998e-05 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.106 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:916 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984413+0000", 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.082679831, 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073160899999999999, 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 916 2024-09-16T11:13:33.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984413+0000", 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984413+0000", 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984414+0000", 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5999999999999991e-07 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984421+0000", 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.832e-06 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984434+0000", 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3192e-05 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984442+0000", 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6059999999999997e-06 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.986810+0000", 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023685989999999999 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.986918+0000", 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010765200000000001 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.987391+0000", 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000473258 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989552+0000", 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021602890000000001 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991602+0000", 2024-09-16T11:13:33.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020509130000000001 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991661+0000", 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8295999999999999e-05 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991729+0000", 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8492999999999996e-05 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:917 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984451+0000", 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.082641912, 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0081242050000000007, 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 917 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984451+0000", 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984451+0000", 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984452+0000", 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2500000000000004e-07 2024-09-16T11:13:33.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984457+0000", 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7899999999999999e-06 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984473+0000", 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6183000000000001e-05 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984481+0000", 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.695e-06 2024-09-16T11:13:33.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.988141+0000", 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036604089999999999 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.988234+0000", 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2855000000000005e-05 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.988642+0000", 2024-09-16T11:13:33.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000407649 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991549+0000", 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0029075989999999999 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992507+0000", 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00095825900000000004 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992547+0000", 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9165999999999997e-05 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992575+0000", 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8674999999999999e-05 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:918 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984490+0000", 2024-09-16T11:13:33.118 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.082603198999999, 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082302569999999995, 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 918 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984490+0000", 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984490+0000", 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984491+0000", 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984496+0000", 2024-09-16T11:13:33.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0669999999999999e-06 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984509+0000", 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2736e-05 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984516+0000", 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6960000000000005e-06 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989623+0000", 2024-09-16T11:13:33.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0051062870000000002 2024-09-16T11:13:33.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989734+0000", 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111074 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990137+0000", 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040373900000000002 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991557+0000", 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001419897 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992620+0000", 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00106251 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992671+0000", 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1359000000000001e-05 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992720+0000", 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8881000000000002e-05 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:919 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.984525+0000", 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.082567584, 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094882379999999995, 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.125 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 919 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984525+0000", 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984525+0000", 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984526+0000", 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2399999999999997e-07 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984531+0000", 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6179999999999997e-06 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984567+0000", 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.984575+0000", 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3389999999999993e-06 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990760+0000", 2024-09-16T11:13:33.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061848299999999997 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990836+0000", 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6500000000000003e-05 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.991213+0000", 2024-09-16T11:13:33.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037719499999999998 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.992237+0000", 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001024016 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.993619+0000", 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001381267 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.993816+0000", 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019750799999999999 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.994014+0000", 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000197395 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:920 4.9 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.989540+0000", 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.077552719, 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076473119999999999, 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 920 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989540+0000", 2024-09-16T11:13:33.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989540+0000", 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989543+0000", 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3530000000000002e-06 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989555+0000", 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2646999999999999e-05 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989578+0000", 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2603999999999999e-05 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989591+0000", 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2984e-05 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989706+0000", 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011489699999999999 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.989892+0000", 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000186465 2024-09-16T11:13:33.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.990156+0000", 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026365600000000002 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.995724+0000", 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005567944 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997072+0000", 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001348506 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997134+0000", 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1396999999999995e-05 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997188+0000", 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3859000000000001e-05 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.137 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:930 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.069794441999999, 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080905809999999995, 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.138 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 930 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997299+0000", 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0100000000000004e-07 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997303+0000", 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2559999999999998e-06 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997313+0000", 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0434e-05 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.997317+0000", 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1840000000000001e-06 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.999225+0000", 2024-09-16T11:13:33.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001907883 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.999310+0000", 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4845999999999994e-05 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.999700+0000", 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00039045900000000001 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.003793+0000", 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040925340000000001 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.005311+0000", 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001518482 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.005357+0000", 2024-09-16T11:13:33.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5621999999999998e-05 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.005389+0000", 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2079999999999998e-05 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:1000 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:21.177824+0000", 2024-09-16T11:13:33.144 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.889269244999999, 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010724525, 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1000 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177824+0000", 2024-09-16T11:13:33.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177824+0000", 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177825+0000", 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1969999999999999e-06 2024-09-16T11:13:33.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177831+0000", 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.353e-06 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177848+0000", 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6211000000000001e-05 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.177857+0000", 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9569999999999997e-06 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.182271+0000", 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044136790000000002 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.182398+0000", 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012694400000000001 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.182908+0000", 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000509667 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.184150+0000", 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00124249 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188479+0000", 2024-09-16T11:13:33.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043282540000000001 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188511+0000", 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2466999999999999e-05 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188548+0000", 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7305999999999998e-05 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:1006 4.1b 4:d9c065bc:::rbd_object_map.11b585795d6b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:21.188332+0000", 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.878760945, 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0074913419999999998, 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1006 2024-09-16T11:13:33.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188332+0000", 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188332+0000", 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188333+0000", 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188336+0000", 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1609999999999999e-06 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188360+0000", 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-05 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.188364+0000", 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0239999999999999e-06 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.189849+0000", 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0014850429999999999 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.189907+0000", 2024-09-16T11:13:33.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8303000000000001e-05 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.190209+0000", 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000301947 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.191731+0000", 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001521957 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.195761+0000", 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040294659999999998 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.195797+0000", 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5918999999999999e-05 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:21.195823+0000", 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6659999999999999e-05 2024-09-16T11:13:33.157 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2095 4.1b 4:ddcc03d7:::rbd_data.11b51c0df580.0000000000000000:7 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.526746+0000", 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.5403469699999999, 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007442387, 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2095 2024-09-16T11:13:33.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526746+0000", 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526746+0000", 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526747+0000", 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9999999999999997e-07 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526749+0000", 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7530000000000002e-06 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526758+0000", 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0770000000000008e-06 2024-09-16T11:13:33.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526763+0000", 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1100000000000002e-06 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.533931+0000", 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0071678949999999996 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534006+0000", 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5722000000000004e-05 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534188+0000", 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018212999999999999 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2096 4.1b 4:ddcc03d7:::rbd_data.11b51c0df580.0000000000000000:7 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.162 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.526768+0000", 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.5403249730000006, 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076987260000000004, 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2096 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526768+0000", 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526768+0000", 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526769+0000", 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999998e-07 2024-09-16T11:13:33.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526771+0000", 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5399999999999998e-06 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526779+0000", 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0970000000000006e-06 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526785+0000", 2024-09-16T11:13:33.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2109999999999997e-06 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534312+0000", 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075269769999999998 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534373+0000", 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1636000000000004e-05 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534467+0000", 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3561999999999996e-05 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2097 4.1b 4:ddcc03d7:::rbd_data.11b51c0df580.0000000000000000:7 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.526790+0000", 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.5403030819999994, 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078954669999999998, 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.167 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2097 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526790+0000", 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526790+0000", 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526791+0000", 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0599999999999999e-07 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526793+0000", 2024-09-16T11:13:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5940000000000001e-06 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526803+0000", 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8330000000000006e-06 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.526809+0000", 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7409999999999998e-06 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534503+0000", 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076938589999999999 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534585+0000", 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2422999999999993e-05 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534685+0000", 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100111 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2184 4.9 4:93045ea3:::rbd_data.11b5220ac4a2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.532860+0000", 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.5342325680000002, 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098338330000000002, 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2184 2024-09-16T11:13:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.532860+0000", 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.532860+0000", 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.532862+0000", 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.658e-06 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534381+0000", 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001519095 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534395+0000", 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4053000000000001e-05 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534405+0000", 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.012e-05 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535334+0000", 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00092820100000000003 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535496+0000", 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016220899999999999 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535870+0000", 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037407100000000002 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539197+0000", 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033272689999999999 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.542561+0000", 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033639360000000001 2024-09-16T11:13:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.542641+0000", 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9744000000000001e-05 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.542694+0000", 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3477000000000003e-05 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:3939 4.1a 4:59e28139:::rbd_data.11b5b93336d2.000000000000001e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:28.667389+0000", 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.3997039449999997, 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080127329999999993, 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 3939 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667389+0000", 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667389+0000", 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667391+0000", 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.323e-06 2024-09-16T11:13:33.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667403+0000", 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1219999999999999e-05 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667435+0000", 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2564999999999997e-05 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667449+0000", 2024-09-16T11:13:33.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3963e-05 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667548+0000", 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8692999999999999e-05 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667684+0000", 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013601999999999999 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.667846+0000", 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016197800000000001 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.669208+0000", 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001361952 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.675143+0000", 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059356290000000004 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.675283+0000", 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139937 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.675402+0000", 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118453 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.183 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:4001 4.1b 4:dba71535:::rbd_data.11b5b93336d2.0000000000000016:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.315673425, 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.027255873, 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 4001 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751422+0000", 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2469999999999998e-06 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751434+0000", 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1761e-05 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751469+0000", 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5577000000000002e-05 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751483+0000", 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3570999999999999e-05 2024-09-16T11:13:33.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751628+0000", 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014536400000000001 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751857+0000", 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000228452 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.758485+0000", 2024-09-16T11:13:33.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066286000000000001 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778412+0000", 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019927027 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778513+0000", 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010084 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778611+0000", 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8432999999999998e-05 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778675+0000", 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4000999999999999e-05 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:4170 4.1 4:8514a423:::rbd_data.11b5b93336d2.0000000000000059:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:29.161302+0000", 2024-09-16T11:13:33.189 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9057905509999999, 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011719693999999999, 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 4170 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161302+0000", 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161302+0000", 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161305+0000", 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.12e-06 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161316+0000", 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0937e-05 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161350+0000", 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4034e-05 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161362+0000", 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2874000000000001e-05 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161481+0000", 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000119083 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.161745+0000", 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000263906 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.163591+0000", 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001845675 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.172728+0000", 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091370110000000004 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.172858+0000", 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012993 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.172961+0000", 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102699 2024-09-16T11:13:33.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:29.173022+0000", 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1425000000000002e-05 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.195 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:33.196 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:33.196 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:33.197 INFO:teuthology.orchestra.run.smithi162.stdout:1059 trunc from 0xba94255 to 0x4ad04ca 2024-09-16T11:13:33.197 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:33.198 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:33.198 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:33.198 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:33.198 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:33.199 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:33.199 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:33.199 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:33.199 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:33.199 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:33.199 INFO:teuthology.orchestra.run.smithi162.stdout:1060 read 0x65a8e1 thru 0x65bff0 (0x1710 bytes) 2024-09-16T11:13:33.200 INFO:teuthology.orchestra.run.smithi162.stdout:1061 write 0xcc20087 thru 0xcc23641 (0x35bb bytes) 2024-09-16T11:13:33.200 INFO:teuthology.orchestra.run.smithi162.stdout:1062 punch from 0x902b8f2 to 0x90351d9, (0x98e7 bytes) 2024-09-16T11:13:33.200 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.200 INFO:teuthology.orchestra.run.smithi162.stdout:1065 trunc from 0xcc23642 to 0x199f7ee 2024-09-16T11:13:33.201 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:33.189+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:33.201 INFO:teuthology.orchestra.run.smithi162.stdout:1066 read 0x652382 thru 0x66099f (0xe61e bytes) 2024-09-16T11:13:33.202 INFO:teuthology.orchestra.run.smithi162.stdout:1067 writesame 0x15457b1 thru 0x1549e49 data_size 0x1f(0x4699 bytes) 2024-09-16T11:13:33.237 INFO:teuthology.orchestra.run.smithi162.stdout:1068 write 0xb2f497d thru 0xb2fd533 (0x8bb7 bytes) 2024-09-16T11:13:33.240 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:13:33.259 INFO:teuthology.orchestra.run.smithi162.stdout:1069 flatten 2024-09-16T11:13:33.268 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.284 INFO:teuthology.orchestra.run.smithi162.stdout:1075 write 0xe56ef27 thru 0xe57049d (0x1577 bytes) 2024-09-16T11:13:33.298 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:33.293+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:33.302 INFO:teuthology.orchestra.run.smithi162.stdout:1076 write 0xe964bf9 thru 0xe96ed50 (0xa158 bytes) 2024-09-16T11:13:33.310 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:33.310 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:13486 4.8 4:159b0278:::rbd_data.11b52f8117af.00000000000000e0:head [set-alloc-hint object_size 1048576 write_size 1048576,write 704512~4096 in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:33.307853+0000", 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.00050783800000000004, 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00055971899999999997, 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 13486 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.307853+0000", 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.307853+0000", 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.307856+0000", 2024-09-16T11:13:33.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8849999999999999e-06 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.307878+0000", 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2035000000000002e-05 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.307915+0000", 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6471999999999999e-05 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.307929+0000", 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.414e-05 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.308031+0000", 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010242399999999999 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.308248+0000", 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021708099999999999 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.314 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.315 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.315 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.315 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:13:33.315 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 1 2024-09-16T11:13:33.315 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:33.321 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:33.331 INFO:teuthology.orchestra.run.smithi162.stdout:1077 trunc from 0xe96ed51 to 0x72de8e3 2024-09-16T11:13:33.353 INFO:teuthology.orchestra.run.smithi162.stdout:1078 write 0x108c75f thru 0x1094268 (0x7b0a bytes) 2024-09-16T11:13:33.383 INFO:teuthology.orchestra.run.smithi162.stdout:1079 write 0xd78b38f thru 0xd797011 (0xbc83 bytes) 2024-09-16T11:13:33.398 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:33.393+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:33.404 INFO:teuthology.orchestra.run.smithi162.stdout:1080 write 0x7c4ddc2 thru 0x7c5b9f7 (0xdc36 bytes) 2024-09-16T11:13:33.424 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:33.424 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:33.427 INFO:teuthology.orchestra.run.smithi162.stdout:1081 write 0x11d0f16 thru 0x11d9849 (0x8934 bytes) 2024-09-16T11:13:33.432 INFO:teuthology.orchestra.run.smithi162.stdout:1082 write 0x704a206 thru 0x70530dd (0x8ed8 bytes) 2024-09-16T11:13:33.435 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:33.436 INFO:teuthology.orchestra.run.smithi162.stdout:1084 punch from 0xc761084 to 0xc76e6ac, (0xd628 bytes) 2024-09-16T11:13:33.436 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.454 INFO:teuthology.orchestra.run.smithi162.stdout:1090 trunc from 0xd797012 to 0x1e7438c 2024-09-16T11:13:33.486 INFO:teuthology.orchestra.run.smithi162.stdout:1092 trunc from 0x1e7438c to 0x8f67ce8 2024-09-16T11:13:33.495 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.496 INFO:teuthology.orchestra.run.smithi162.stdout:1094 read 0x50232c6 thru 0x5028425 (0x5160 bytes) 2024-09-16T11:13:33.499 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:33.497+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:33.501 INFO:teuthology.orchestra.run.smithi162.stdout:1098 trunc from 0x8f67ce8 to 0x54dfad9 2024-09-16T11:13:33.513 INFO:teuthology.orchestra.run.smithi162.stdout:1099 trunc from 0x54dfad9 to 0x5aa6856 2024-09-16T11:13:33.548 INFO:teuthology.orchestra.run.smithi162.stdout:1101 write 0xd02af3e thru 0xd034c9c (0x9d5f bytes) 2024-09-16T11:13:33.575 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.575 INFO:teuthology.orchestra.run.smithi162.stdout:1103 write 0x98e73b6 thru 0x98f229d (0xaee8 bytes) 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4276.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.754185790999998, 2024-09-16T11:13:33.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091644129999999997, 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4276", 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/2834747430", 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822526+0000", 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822531+0000", 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1739999999999999e-06 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822539+0000", 2024-09-16T11:13:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4050000000000003e-06 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822554+0000", 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5384e-05 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822564+0000", 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7510000000000003e-06 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822695+0000", 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131481 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.822720+0000", 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5069999999999999e-05 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826480+0000", 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037597590000000001 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826680+0000", 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000200304 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 2", 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.826846+0000", 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016532299999999999 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.828757+0000", 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001911598 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831554+0000", 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027972930000000002 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831613+0000", 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8715000000000003e-05 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:54.831690+0000", 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7156000000000005e-05 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:594 4.8 4:125c4097:::rbd_object_map.11b52605a025.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301603+0000", 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.275108813999999, 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0068584780000000003, 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 594 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301603+0000", 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301603+0000", 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301604+0000", 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301608+0000", 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4050000000000002e-06 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301616+0000", 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1040000000000007e-06 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301623+0000", 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303230+0000", 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0016072149999999999 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303314+0000", 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3752999999999996e-05 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303660+0000", 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034616399999999999 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308254+0000", 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004593728 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308372+0000", 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117806 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308423+0000", 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.147e-05 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308461+0000", 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.841e-05 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:602 4.8 4:125c4097:::rbd_object_map.11b52605a025.0000000000000005:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303307+0000", 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.273404571, 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0079281130000000005, 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 602 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303307+0000", 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303307+0000", 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303308+0000", 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0699999999999996e-07 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303313+0000", 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0470000000000001e-06 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303325+0000", 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1574e-05 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303332+0000", 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3039999999999999e-06 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307030+0000", 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036981510000000002 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307138+0000", 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010738000000000001 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307546+0000", 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040797799999999998 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311124+0000", 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035785550000000002 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311132+0000", 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9570000000000002e-06 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311194+0000", 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2286999999999995e-05 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311235+0000", 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0973000000000002e-05 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:615 4.15 4:ad422a37:::rbd_data.11b52605a025.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.264798227, 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010270354000000001, 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 615 2024-09-16T11:13:33.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311915+0000", 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1969999999999999e-06 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311920+0000", 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0010000000000001e-06 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311933+0000", 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2948e-05 2024-09-16T11:13:33.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311937+0000", 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8219999999999997e-06 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311999+0000", 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1174999999999996e-05 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312158+0000", 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015924900000000001 2024-09-16T11:13:33.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312292+0000", 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013398899999999999 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313867+0000", 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001575087 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322101+0000", 2024-09-16T11:13:33.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082335659999999995 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322144+0000", 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3900000000000003e-05 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322184+0000", 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9419999999999999e-05 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 12.713757979, 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:13:33.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2048 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3276800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.610 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524324+0000", 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0523881520000007, 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070262839999999998, 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2048 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524324+0000", 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524324+0000", 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524325+0000", 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0419999999999999e-06 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524337+0000", 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2853999999999999e-05 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524351+0000", 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3552e-05 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524363+0000", 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:33.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531097+0000", 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006733859 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531200+0000", 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102729 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531350+0000", 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015020499999999999 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2066 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 393216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524379+0000", 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0523329829999994, 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072794260000000003, 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2066 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524379+0000", 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524379+0000", 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524380+0000", 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1200000000000001e-07 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524384+0000", 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5279999999999997e-06 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524397+0000", 2024-09-16T11:13:33.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3083e-05 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524408+0000", 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1131000000000001e-05 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531392+0000", 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069834820000000001 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531522+0000", 2024-09-16T11:13:33.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013043700000000001 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531658+0000", 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013585299999999999 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2067 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1572864~39333] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524422+0000", 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0522893870000001, 2024-09-16T11:13:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098132879999999999, 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2067 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524422+0000", 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524422+0000", 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524423+0000", 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1999999999999998e-07 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524428+0000", 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5179999999999998e-06 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524441+0000", 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.348e-05 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524450+0000", 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4330000000000004e-06 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531701+0000", 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0072510020000000003 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.531814+0000", 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011318400000000001 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534236+0000", 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024217510000000002 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2068 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.623 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0522341950000005, 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099591620000000006, 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2068 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:33.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524479+0000", 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.155e-06 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524483+0000", 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5680000000000001e-06 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524497+0000", 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4018e-05 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524509+0000", 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1735e-05 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534308+0000", 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097988120000000005 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534360+0000", 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2094e-05 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534437+0000", 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6779999999999993e-05 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2069 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0521862080000002, 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010200859, 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2069 2024-09-16T11:13:33.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524527+0000", 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.02e-06 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524531+0000", 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4900000000000002e-06 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524544+0000", 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.311e-05 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524552+0000", 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6879999999999999e-06 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534481+0000", 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099292119999999998 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534629+0000", 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148281 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534726+0000", 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7058000000000005e-05 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2081 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525000+0000", 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0517113069999997, 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009893542, 2024-09-16T11:13:33.632 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2081 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525000+0000", 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525000+0000", 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525003+0000", 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8090000000000001e-06 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525014+0000", 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1083e-05 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525040+0000", 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5925999999999999e-05 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525057+0000", 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6293000000000001e-05 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534751+0000", 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009694055 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534813+0000", 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2318999999999997e-05 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534894+0000", 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1057000000000007e-05 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.636 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2082 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0516391049999996, 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010054116, 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2082 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525074+0000", 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4470000000000001e-06 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525080+0000", 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8980000000000001e-06 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525099+0000", 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8960000000000001e-05 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525112+0000", 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3013999999999999e-05 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534937+0000", 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098250449999999993 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535013+0000", 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6108999999999999e-05 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535127+0000", 2024-09-16T11:13:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113643 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2083 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0515852760000008, 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010171747, 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.641 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2083 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525128+0000", 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.082e-06 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525134+0000", 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8079999999999999e-06 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525149+0000", 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4833e-05 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525158+0000", 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5820000000000002e-06 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535171+0000", 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010012828999999999 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535224+0000", 2024-09-16T11:13:33.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3826999999999998e-05 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535298+0000", 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3786000000000001e-05 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2098 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:33.645 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0514733449999998, 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010231951, 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2098 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525240+0000", 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.494e-06 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525247+0000", 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7710000000000001e-06 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525265+0000", 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7966e-05 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525276+0000", 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1827e-05 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535332+0000", 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010055633 2024-09-16T11:13:33.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535388+0000", 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5741e-05 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535470+0000", 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2519000000000001e-05 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2099 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0512592830000003, 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010173994, 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2099 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525455+0000", 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6469999999999998e-06 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525468+0000", 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2931e-05 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525518+0000", 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0448000000000002e-05 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525542+0000", 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3870999999999999e-05 2024-09-16T11:13:33.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535510+0000", 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099681400000000003 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535559+0000", 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8028999999999997e-05 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535626+0000", 2024-09-16T11:13:33.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7928000000000004e-05 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2100 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0511510519999998, 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010216233999999999, 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2100 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525562+0000", 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3030000000000001e-06 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525569+0000", 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9240000000000001e-06 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525596+0000", 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7430999999999999e-05 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525645+0000", 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8597000000000002e-05 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535663+0000", 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010018132000000001 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535711+0000", 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7706e-05 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535777+0000", 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6141000000000003e-05 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2101 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.658 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0510515890000001, 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010279129999999999, 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2101 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:33.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525662+0000", 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.437e-06 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525668+0000", 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0900000000000001e-06 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525709+0000", 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1464000000000001e-05 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525726+0000", 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7210999999999998e-05 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535807+0000", 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010080621999999999 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535872+0000", 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5504000000000002e-05 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535939+0000", 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6802000000000002e-05 2024-09-16T11:13:33.662 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2266 4.8 4:11824548:::rbd_data.11b5220ac4a2.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.538416+0000", 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0382960810000004, 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0070944470000000003, 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.663 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2266 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.538416+0000", 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.538416+0000", 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.538418+0000", 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.277e-06 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539392+0000", 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00097422399999999999 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539413+0000", 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0610000000000001e-05 2024-09-16T11:13:33.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539421+0000", 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0139999999999998e-06 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539482+0000", 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1396999999999995e-05 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539606+0000", 2024-09-16T11:13:33.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012412400000000001 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.539817+0000", 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000210745 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.543132+0000", 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033147290000000002 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.545419+0000", 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022871559999999998 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.545470+0000", 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1020000000000003e-05 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.545510+0000", 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0151000000000002e-05 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.668 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2720 4.15 4:a82fda07:::rbd_data.11b5220ac4a2.0000000000000026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:25.388510+0000", 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1882016380000007, 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075686929999999996, 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2720 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388510+0000", 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388510+0000", 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388511+0000", 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:13:33.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388515+0000", 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5370000000000002e-06 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388534+0000", 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9040000000000001e-05 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388540+0000", 2024-09-16T11:13:33.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6499999999999999e-06 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.389992+0000", 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001451379 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.390446+0000", 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045423700000000002 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:33.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.390831+0000", 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038509200000000002 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.393896+0000", 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003064893 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.395936+0000", 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0020395560000000001 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396020+0000", 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4237e-05 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396079+0000", 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8998e-05 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2723 4.15 4:ac84a83f:::rbd_data.11b5220ac4a2.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:25.388715+0000", 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1879972250000002, 2024-09-16T11:13:33.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007568155, 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2723 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388715+0000", 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388715+0000", 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388716+0000", 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.232e-06 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388722+0000", 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4230000000000002e-06 2024-09-16T11:13:33.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388739+0000", 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-05 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.388762+0000", 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3482999999999999e-05 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.391820+0000", 2024-09-16T11:13:33.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0030575260000000001 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.392042+0000", 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022243999999999999 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.392205+0000", 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016326900000000001 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.395502+0000", 2024-09-16T11:13:33.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0032966580000000001 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396159+0000", 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00065711999999999997 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396229+0000", 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0193999999999999e-05 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:25.396283+0000", 2024-09-16T11:13:33.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.321e-05 2024-09-16T11:13:33.680 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.680 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.680 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.680 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:33.680 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:33.680 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:1105 punch from 0x5fc598e to 0x5fcd14f, (0x77c1 bytes) 2024-09-16T11:13:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:1106 trunc from 0xd034c9d to 0x75fae07 2024-09-16T11:13:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:1108 trunc from 0x75fae07 to 0xd7d529b 2024-09-16T11:13:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:1109 writesame 0xd02dc04 thru 0xd03a809 data_size 0xf(0xcc06 bytes) 2024-09-16T11:13:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:1110 read 0x9b75eba thru 0x9b7a73f (0x4886 bytes) 2024-09-16T11:13:33.682 INFO:teuthology.orchestra.run.smithi162.stdout:1111 read 0x11f15ec thru 0x1200446 (0xee5b bytes) 2024-09-16T11:13:33.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:33.683 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:33.677+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:33.683 INFO:teuthology.orchestra.run.smithi162.stdout:1112 writesame 0xc874d4a thru 0xc875ada data_size 0x1(0xd91 bytes) 2024-09-16T11:13:33.698 INFO:teuthology.orchestra.run.smithi162.stdout:1113 writesame 0xe9f29b8 thru 0xe9fbf8e data_size 0x1(0x95d7 bytes) 2024-09-16T11:13:33.698 INFO:teuthology.orchestra.run.smithi162.stdout:1114 read 0x4cf7b04 thru 0x4cfb5b2 (0x3aaf bytes) 2024-09-16T11:13:33.712 INFO:teuthology.orchestra.run.smithi162.stdout:1115 trunc from 0xe9fbf8f to 0x7782afe 2024-09-16T11:13:33.754 INFO:teuthology.orchestra.run.smithi162.stdout:1116 writesame 0xa003fbd thru 0xa00a25c data_size 0x30(0x62a0 bytes) 2024-09-16T11:13:33.754 INFO:teuthology.orchestra.run.smithi162.stdout:1117 read 0x1b7277e thru 0x1b78b5e (0x63e1 bytes) 2024-09-16T11:13:33.755 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.755 INFO:teuthology.orchestra.run.smithi162.stdout:1119 write 0x76211c8 thru 0x7624700 (0x3539 bytes) 2024-09-16T11:13:33.766 INFO:teuthology.orchestra.run.smithi162.stdout:1120 punch from 0x15a087a to 0x15b0116, (0xf89c bytes) 2024-09-16T11:13:33.766 INFO:teuthology.orchestra.run.smithi162.stdout:1121 writesame 0x400bd4f thru 0x400c33f data_size 0xd(0x5f1 bytes) 2024-09-16T11:13:33.773 INFO:teuthology.orchestra.run.smithi162.stdout:1122 read 0x52d0218 thru 0x52d20ee (0x1ed7 bytes) 2024-09-16T11:13:33.782 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:33.777+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:33.790 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:33.791 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:33.793 INFO:teuthology.orchestra.run.smithi162.stdout:1123 write 0xdd4c74d thru 0xdd5bc60 (0xf514 bytes) 2024-09-16T11:13:33.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:33.821 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.821 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.822 INFO:teuthology.orchestra.run.smithi162.stdout:1129 write 0xaf96092 thru 0xaf96ad4 (0xa43 bytes) 2024-09-16T11:13:33.825 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.849 INFO:teuthology.orchestra.run.smithi162.stdout:1131 trunc from 0xdd5bc61 to 0x1eb9dd1 2024-09-16T11:13:33.872 INFO:teuthology.orchestra.run.smithi162.stdout:1134 write 0xfa1a9c thru 0xfacb4e (0xb0b3 bytes) 2024-09-16T11:13:33.883 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.883 INFO:teuthology.orchestra.run.smithi162.stdout:1137 read 0x1ab6b67 thru 0x1ac0e02 (0xa29c bytes) 2024-09-16T11:13:33.883 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:33.881+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:33.884 INFO:teuthology.orchestra.run.smithi162.stdout:1138 punch from 0x150d4a2 to 0x151c459, (0xefb7 bytes) 2024-09-16T11:13:33.884 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.884 INFO:teuthology.orchestra.run.smithi162.stdout:1140 read 0x148a61c thru 0x148f3cc (0x4db1 bytes) 2024-09-16T11:13:33.884 INFO:teuthology.orchestra.run.smithi162.stdout:1141 read 0x15e8237 thru 0x15f19fe (0x97c8 bytes) 2024-09-16T11:13:33.897 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:33.897 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:33.906 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:33.916 INFO:teuthology.orchestra.run.smithi162.stdout:1142 write 0xb718c40 thru 0xb7278a4 (0xec65 bytes) 2024-09-16T11:13:33.945 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:33.945 INFO:teuthology.orchestra.run.smithi162.stdout:1144 read 0x654b639 thru 0x6559b57 (0xe51f bytes) 2024-09-16T11:13:33.945 INFO:teuthology.orchestra.run.smithi162.stdout:1145 read 0x73df0f8 thru 0x73e7a2d (0x8936 bytes) 2024-09-16T11:13:33.960 INFO:teuthology.orchestra.run.smithi162.stdout:1146 write 0xe40eac6 thru 0xe41a662 (0xbb9d bytes) 2024-09-16T11:13:33.984 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:33.981+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:34.011 INFO:teuthology.orchestra.run.smithi162.stdout:1147 trunc from 0xe41a663 to 0x19374b1 2024-09-16T11:13:34.014 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.878576458000001, 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066095169999999996, 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 11 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130361+0000", 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130364+0000", 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2200000000000001e-06 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130378+0000", 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3498e-05 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130415+0000", 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6928999999999998e-05 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130431+0000", 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5834000000000001e-05 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130591+0000", 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016076400000000001 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.130610+0000", 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8244999999999999e-05 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.133935+0000", 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033251650000000002 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134174+0000", 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023965400000000001 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.134553+0000", 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037902299999999998 2024-09-16T11:13:34.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136098+0000", 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0015447010000000001 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136814+0000", 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00071608299999999995 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136888+0000", 2024-09-16T11:13:34.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3526e-05 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:15.136971+0000", 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2874999999999995e-05 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:544 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.711561951, 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073024969999999998, 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 544 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297376+0000", 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297378+0000", 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.99e-06 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297388+0000", 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0407e-05 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297415+0000", 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7498e-05 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297426+0000", 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0957e-05 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297594+0000", 2024-09-16T11:13:34.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167157 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297849+0000", 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025576 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.298833+0000", 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098353599999999992 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300723+0000", 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001890395 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304585+0000", 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038620709999999999 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304639+0000", 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3193999999999999e-05 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304678+0000", 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9532e-05 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:549 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.711133577, 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.007009576, 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 549 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297804+0000", 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297806+0000", 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.747e-06 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297814+0000", 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2900000000000002e-06 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297858+0000", 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4295000000000002e-05 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.297868+0000", 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2639999999999997e-06 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299859+0000", 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019912609999999998 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.299944+0000", 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5069999999999997e-05 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300312+0000", 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036818899999999999 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304727+0000", 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0044150689999999998 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304734+0000", 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304779+0000", 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5055000000000001e-05 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304814+0000", 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4770000000000001e-05 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.033 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:591 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.708339789, 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043393090000000004, 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 591 2024-09-16T11:13:34.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300598+0000", 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300600+0000", 2024-09-16T11:13:34.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1320000000000001e-06 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300611+0000", 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0961000000000001e-05 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300633+0000", 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1999000000000001e-05 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300644+0000", 2024-09-16T11:13:34.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1632e-05 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300770+0000", 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012577899999999999 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300927+0000", 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000157186 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301666+0000", 2024-09-16T11:13:34.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00073808200000000004 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304821+0000", 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0031557349999999998 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304859+0000", 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7450999999999998e-05 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304902+0000", 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3579999999999999e-05 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304937+0000", 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4771999999999998e-05 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:592 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.708058949, 2024-09-16T11:13:34.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0061751929999999998, 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 592 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300879+0000", 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300880+0000", 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6080000000000001e-06 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300890+0000", 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4040000000000001e-06 2024-09-16T11:13:34.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300907+0000", 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6903000000000001e-05 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.300916+0000", 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5219999999999996e-06 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302514+0000", 2024-09-16T11:13:34.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00159807 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302687+0000", 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000173155 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303339+0000", 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0006514 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306894+0000", 2024-09-16T11:13:34.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035556720000000002 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306940+0000", 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5893999999999998e-05 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307005+0000", 2024-09-16T11:13:34.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4432999999999998e-05 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307054+0000", 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9132000000000003e-05 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:593 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.707753795, 2024-09-16T11:13:34.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082575289999999996, 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 593 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301184+0000", 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301185+0000", 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2860000000000001e-06 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301193+0000", 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.678e-06 2024-09-16T11:13:34.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301209+0000", 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5797000000000001e-05 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301218+0000", 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1980000000000007e-06 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303958+0000", 2024-09-16T11:13:34.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027405810000000002 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304015+0000", 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7074000000000003e-05 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304249+0000", 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023344199999999999 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307114+0000", 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028655569999999999 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309341+0000", 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022266339999999999 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309401+0000", 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0436000000000002e-05 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309441+0000", 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9845999999999998e-05 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:595 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.707702878999999, 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010351252, 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 595 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301235+0000", 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2999999999999999e-07 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301240+0000", 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8359999999999999e-06 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301269+0000", 2024-09-16T11:13:34.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8459e-05 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301278+0000", 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7949999999999996e-06 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.304969+0000", 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003691864 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305136+0000", 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016632999999999999 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305576+0000", 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043990000000000001 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309262+0000", 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0036865930000000002 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311524+0000", 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022620909999999999 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311559+0000", 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4597000000000001e-05 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311586+0000", 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6957e-05 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.056 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:597 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.707560896, 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010265962, 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 597 2024-09-16T11:13:34.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301377+0000", 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301378+0000", 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.657e-06 2024-09-16T11:13:34.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301387+0000", 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3860000000000007e-06 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301402+0000", 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4875000000000001e-05 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301411+0000", 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5820000000000008e-06 2024-09-16T11:13:34.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305997+0000", 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045854169999999996 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306078+0000", 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1614999999999997e-05 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.306441+0000", 2024-09-16T11:13:34.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036263000000000002 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309329+0000", 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028879650000000002 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311596+0000", 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0022673939999999998 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311621+0000", 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4797e-05 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311643+0000", 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1644000000000001e-05 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:598 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.062 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.707477256000001, 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010230927000000001, 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 598 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301460+0000", 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301462+0000", 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301470+0000", 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6279999999999996e-06 2024-09-16T11:13:34.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301484+0000", 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4314999999999999e-05 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301512+0000", 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8218999999999999e-05 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307142+0000", 2024-09-16T11:13:34.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056298169999999996 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307264+0000", 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121704 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.307701+0000", 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00043725099999999999 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311457+0000", 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037559 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311649+0000", 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019221900000000001 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311673+0000", 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3603000000000001e-05 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311691+0000", 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8609000000000001e-05 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:599 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.70726599, 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014841653999999999, 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 599 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301672+0000", 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301673+0000", 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3459999999999999e-06 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301681+0000", 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0949999999999996e-06 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301696+0000", 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4932e-05 2024-09-16T11:13:34.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.301705+0000", 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6009999999999995e-06 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308183+0000", 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0064783840000000002 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308260+0000", 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6601999999999996e-05 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.308708+0000", 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044877600000000001 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311516+0000", 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028079189999999999 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316392+0000", 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.004875735 2024-09-16T11:13:34.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316467+0000", 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4571000000000001e-05 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316513+0000", 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6693000000000002e-05 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:601 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.706799694000001, 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014436584000000001, 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 601 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302138+0000", 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302139+0000", 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0109999999999999e-06 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302152+0000", 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3608000000000001e-05 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302163+0000", 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1012000000000001e-05 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302169+0000", 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1309999999999997e-06 2024-09-16T11:13:34.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309509+0000", 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0073404869999999997 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.309603+0000", 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4374000000000001e-05 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310124+0000", 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00052012300000000005 2024-09-16T11:13:34.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312552+0000", 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024287969999999999 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316527+0000", 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039741569999999999 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316553+0000", 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5945999999999999e-05 2024-09-16T11:13:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316574+0000", 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1937999999999999e-05 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:603 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.706114253999999, 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019179324000000001, 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 603 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302824+0000", 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.06e-06 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302830+0000", 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9209999999999997e-06 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302844+0000", 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3304e-05 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302850+0000", 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310529+0000", 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076792980000000002 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310601+0000", 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1070999999999998e-05 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311059+0000", 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045884099999999998 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313207+0000", 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002147805 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321862+0000", 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086545330000000007 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321919+0000", 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7130999999999999e-05 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322003+0000", 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3794000000000004e-05 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.084 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:604 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.705308258000001, 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.018458927999999999, 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 604 2024-09-16T11:13:34.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303631+0000", 2024-09-16T11:13:34.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5579999999999999e-06 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303638+0000", 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303654+0000", 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6628000000000001e-05 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303662+0000", 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8390000000000007e-06 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311727+0000", 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080650300000000008 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311786+0000", 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8507000000000001e-05 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312144+0000", 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358821 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.314152+0000", 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002008045 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322013+0000", 2024-09-16T11:13:34.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078606920000000007 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322053+0000", 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322088+0000", 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4972000000000003e-05 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.090 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:605 4.13 e38/31 4:cae7a2e7:::rbd_data.11b52605a025.0000000000000001:head v 38'7, mlcod=35'4)", 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.703376391000001, 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056889410000000003, 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305561+0000", 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305562+0000", 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305567+0000", 2024-09-16T11:13:34.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5949999999999998e-06 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305577+0000", 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0091999999999999e-05 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305584+0000", 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3670000000000002e-06 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305659+0000", 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5593999999999995e-05 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.305672+0000", 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3193000000000001e-05 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311201+0000", 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055283249999999997 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311250+0000", 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9382000000000003e-05 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:613 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'8, mlcod=37'5)", 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.696408871999999, 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095140820000000001, 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312529+0000", 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:13:34.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312549+0000", 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9432999999999998e-05 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312559+0000", 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0455000000000001e-05 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312565+0000", 2024-09-16T11:13:34.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4609999999999999e-06 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312594+0000", 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9456000000000001e-05 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312622+0000", 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7586999999999999e-05 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322004+0000", 2024-09-16T11:13:34.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093818349999999998 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322043+0000", 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9167e-05 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:614 4.13 e38/31 4:cc137bed:::rbd_data.11b52605a025.0000000000000006:head v 38'10, mlcod=37'5)", 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.695917861, 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010261963000000001, 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313020+0000", 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313021+0000", 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5799999999999998e-07 2024-09-16T11:13:34.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313028+0000", 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1400000000000002e-06 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313041+0000", 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2958e-05 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313047+0000", 2024-09-16T11:13:34.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0530000000000003e-06 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313090+0000", 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2936999999999999e-05 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313105+0000", 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5199e-05 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323210+0000", 2024-09-16T11:13:34.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010104958000000001 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323282+0000", 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1860000000000007e-05 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:617 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:34.104 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.692157725, 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0060101130000000001, 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 617 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316780+0000", 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316781+0000", 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.279e-06 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316910+0000", 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012939700000000001 2024-09-16T11:13:34.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316922+0000", 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1894e-05 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316928+0000", 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5559999999999998e-06 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316974+0000", 2024-09-16T11:13:34.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5587000000000002e-05 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317094+0000", 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000120586 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.317242+0000", 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000147517 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322697+0000", 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0054555960000000001 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322708+0000", 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0805000000000001e-05 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322745+0000", 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7280000000000002e-05 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322790+0000", 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4616000000000001e-05 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:618 4.5 4:a70f1c18:::rbd_data.11b52605a025.0000000000000009:head [truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 13.691995868999999, 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.006757379, 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 618 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316942+0000", 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316943+0000", 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9199999999999999e-07 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316945+0000", 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5009999999999999e-06 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316952+0000", 2024-09-16T11:13:34.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5980000000000002e-06 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.316955+0000", 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4879999999999999e-06 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318095+0000", 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0011395050000000001 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318151+0000", 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6577999999999999e-05 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.318284+0000", 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013229499999999999 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323278+0000", 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0049947200000000002 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323605+0000", 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032718800000000001 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323658+0000", 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2961999999999999e-05 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.323699+0000", 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0652000000000003e-05 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2550 4.5 4:a5a2f6f0:::rbd_data.11b5220ac4a2.0000000000000025:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.964379+0000", 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.0445589399999999, 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066632310000000004, 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2550 2024-09-16T11:13:34.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964379+0000", 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964379+0000", 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964382+0000", 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.41e-06 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964408+0000", 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6284000000000001e-05 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964464+0000", 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5875999999999998e-05 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964488+0000", 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4117e-05 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964668+0000", 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017928900000000001 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.964890+0000", 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022197699999999999 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:34.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.965202+0000", 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031292700000000001 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.966789+0000", 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001586495 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.970599+0000", 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0038096250000000001 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.970920+0000", 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320907 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.971042+0000", 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012232399999999999 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:13610 4.5 4:a186f0c5:::rbd_data.11b52f8117af.0000000000000013:head [write 286720~2122] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:33.425769+0000", 2024-09-16T11:13:34.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.583168829, 2024-09-16T11:13:34.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0045657789999999998, 2024-09-16T11:13:34.122 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.122 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.122 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 13610 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425769+0000", 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425769+0000", 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425769+0000", 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2600000000000002e-07 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425773+0000", 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4699999999999998e-06 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425779+0000", 2024-09-16T11:13:34.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1419999999999998e-06 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425782+0000", 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2389999999999998e-06 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425847+0000", 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4449999999999994e-05 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.425971+0000", 2024-09-16T11:13:34.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124315 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 4", 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.426143+0000", 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017206099999999999 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.427411+0000", 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012682730000000001 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.430233+0000", 2024-09-16T11:13:34.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002821465 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.430285+0000", 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2643999999999999e-05 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:33.430334+0000", 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9094000000000001e-05 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.129 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.129 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:34.131 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:34.131 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:34.125+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:34.131 INFO:teuthology.orchestra.run.smithi162.stdout:1153 trunc from 0x19374b1 to 0x7d599b9 2024-09-16T11:13:34.131 INFO:teuthology.orchestra.run.smithi162.stdout:1155 trunc from 0x7d599b9 to 0xe12f2f9 2024-09-16T11:13:34.131 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:34.131 INFO:teuthology.orchestra.run.smithi162.stdout:1157 write 0x873a33c thru 0x8747a44 (0xd709 bytes) 2024-09-16T11:13:34.132 INFO:teuthology.orchestra.run.smithi162.stdout:1159 write 0x9eb7dc4 thru 0x9ebe10d (0x634a bytes) 2024-09-16T11:13:34.132 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:34.132 INFO:teuthology.orchestra.run.smithi162.stdout:1161 read 0xe0dea38 thru 0xe0ec088 (0xd651 bytes) 2024-09-16T11:13:34.132 INFO:teuthology.orchestra.run.smithi162.stdout:1162 write 0x9cd3feb thru 0x9cda105 (0x611b bytes) 2024-09-16T11:13:34.144 INFO:teuthology.orchestra.run.smithi162.stdout:1163 read 0x9f9a41b thru 0x9f9d204 (0x2dea bytes) 2024-09-16T11:13:34.145 INFO:teuthology.orchestra.run.smithi162.stdout:1167 punch from 0xa081914 to 0xa08ab8e, (0x927a bytes) 2024-09-16T11:13:34.145 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:34.145 INFO:teuthology.orchestra.run.smithi162.stdout:1169 writesame 0x2c91fe4 thru 0x2c957a9 data_size 0x16(0x37c6 bytes) 2024-09-16T11:13:34.157 INFO:teuthology.orchestra.run.smithi162.stdout:1170 trunc from 0xe12f2f9 to 0xe7c2a1b 2024-09-16T11:13:34.169 INFO:teuthology.orchestra.run.smithi162.stdout:1171 write 0x965b616 thru 0x9663935 (0x8320 bytes) 2024-09-16T11:13:34.193 INFO:teuthology.orchestra.run.smithi162.stdout:1172 write 0x30fe765 thru 0x3104280 (0x5b1c bytes) 2024-09-16T11:13:34.202 INFO:teuthology.orchestra.run.smithi162.stdout:1174 clone 7 order 18 su 65536 sc 9 2024-09-16T11:13:34.231 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:34.229+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:34.240 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:34.241 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:34.251 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:34.252 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:13:34.332 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:34.325+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:34.349 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:34.349 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:34.349 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:34.349 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:34.349 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:34.359 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:34.433 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:34.425+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:34.453 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:34.453 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.673135205999998, 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:34.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:34.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:34.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.461 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.285209506000001, 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:34.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:34.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.468 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.285058176, 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.469 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:34.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.285006690000003, 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:34.477 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:34.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:34.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.483 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.284936541999997, 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:34.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:34.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.284764072000002, 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:34.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:34.496 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.284705750000001, 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:34.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:34.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:34.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.284553629999998, 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:34.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:34.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.284375111000003, 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.509 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:34.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:34.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.284342138, 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.515 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:34.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.732176645999999, 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:34.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:34.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:34.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:34.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.732154413000004, 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:34.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.535 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.732025616000001, 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.731967271000002, 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.541 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:34.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:34.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.731912882000003, 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:34.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:34.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 43.731778575, 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:34.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:34.558 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.950681054999997, 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:34.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.950623206000003, 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.564 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:34.569 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.950535893999998, 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.570 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:34.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.575 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.950297765999998, 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:34.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:34.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:34.581 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:34.582 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T11:13:34.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:34.584 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:34.577+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:34.684 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:34.677+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:34.691 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:34.692 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:34.692 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:34.692 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:34.701 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:34.760 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:34.760 INFO:teuthology.orchestra.run.smithi162.stdout:1178 read 0x646a573 thru 0x646e701 (0x418f bytes) 2024-09-16T11:13:34.767 INFO:teuthology.orchestra.run.smithi162.stdout:1179 read 0xe7561c3 thru 0xe75beaa (0x5ce8 bytes) 2024-09-16T11:13:34.767 INFO:teuthology.orchestra.run.smithi162.stdout:1180 write 0x6d1d43e thru 0x6d2a166 (0xcd29 bytes) 2024-09-16T11:13:34.780 INFO:teuthology.orchestra.run.smithi162.stdout:1181 write 0xe258933 thru 0xe25bbb3 (0x3281 bytes) 2024-09-16T11:13:34.785 INFO:teuthology.orchestra.run.smithi162.stdout:1182 read 0x6768354 thru 0x676eb05 (0x67b2 bytes) 2024-09-16T11:13:34.785 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:34.781+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:34.785 INFO:teuthology.orchestra.run.smithi162.stdout:1183 writesame 0x1adf6ba thru 0x1ae0645 data_size 0x4(0xf8c bytes) 2024-09-16T11:13:34.798 INFO:teuthology.orchestra.run.smithi162.stdout:1184 read 0x1cc12f7 thru 0x1cc64cf (0x51d9 bytes) 2024-09-16T11:13:34.799 INFO:teuthology.orchestra.run.smithi162.stdout:1186 write 0x9c3f3d thru 0x9cc5e7 (0x86ab bytes) 2024-09-16T11:13:34.808 INFO:teuthology.orchestra.run.smithi162.stdout:1187 read 0x6dbe8d0 thru 0x6dca579 (0xbcaa bytes) 2024-09-16T11:13:34.813 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:34.813 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:34.815 INFO:teuthology.orchestra.run.smithi162.stdout:1188 trunc from 0xe7c2a1b to 0xa34a039 2024-09-16T11:13:34.826 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:34.858 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:34.858 INFO:teuthology.orchestra.run.smithi162.stdout:1193 write 0x5ed9f97 thru 0x5ee6215 (0xc27f bytes) 2024-09-16T11:13:34.863 INFO:teuthology.orchestra.run.smithi162.stdout:1194 read 0x9bf58e0 thru 0x9bfe43c (0x8b5d bytes) 2024-09-16T11:13:34.870 INFO:teuthology.orchestra.run.smithi162.stdout:1195 trunc from 0xa34a039 to 0xb973c39 2024-09-16T11:13:34.884 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:34.885 INFO:teuthology.orchestra.run.smithi162.stdout:1199 write 0xb0caa72 thru 0xb0cb031 (0x5c0 bytes) 2024-09-16T11:13:34.886 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:34.881+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:34.890 INFO:teuthology.orchestra.run.smithi162.stdout:1201 write 0x98272e5 thru 0x983062e (0x934a bytes) 2024-09-16T11:13:34.906 INFO:teuthology.orchestra.run.smithi162.stdout:1202 trunc from 0xb973c39 to 0xe46076b 2024-09-16T11:13:34.917 INFO:teuthology.orchestra.run.smithi162.stdout:1204 read 0x9161d62 thru 0x9168baf (0x6e4e bytes) 2024-09-16T11:13:34.918 INFO:teuthology.orchestra.run.smithi162.stdout:1205 punch from 0x7c54148 to 0x7c54be2, (0xa9a bytes) 2024-09-16T11:13:34.919 INFO:teuthology.orchestra.run.smithi162.stdout:1206 punch from 0x4da16e6 to 0x4daaaa2, (0x93bc bytes) 2024-09-16T11:13:34.919 INFO:teuthology.orchestra.run.smithi162.stdout:1207 write 0x7e9681 thru 0x7e96b0 (0x30 bytes) 2024-09-16T11:13:34.935 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:34.935 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:34.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:34.935 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:34.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.935 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:756 4.b 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.147650162, 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035717454000000003, 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.936 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 756 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:34.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784871+0000", 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.497e-06 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790900+0000", 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060292039999999998 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790939+0000", 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8844999999999998e-05 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790951+0000", 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2143999999999999e-05 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791017+0000", 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5969999999999993e-05 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791210+0000", 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000192993 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791635+0000", 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042530200000000002 2024-09-16T11:13:34.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.812994+0000", 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021359210999999999 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820448+0000", 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0074540709999999996 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:34.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820509+0000", 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0625999999999999e-05 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820586+0000", 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6590999999999999e-05 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2730 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.402039+0000", 2024-09-16T11:13:34.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.5304795250000005, 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013361654000000001, 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2730 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402039+0000", 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402039+0000", 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402040+0000", 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5099999999999998e-07 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402043+0000", 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.286e-06 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402055+0000", 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.17e-05 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402060+0000", 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9309999999999999e-06 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.407941+0000", 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058809550000000002 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.408095+0000", 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015458 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.408767+0000", 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00067182899999999998 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.410676+0000", 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00190877 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.415278+0000", 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004602755 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.415344+0000", 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5782000000000001e-05 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.415400+0000", 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6215000000000003e-05 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15056 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 716800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103515282, 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099743899999999996, 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15056 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.951 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:34.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829006+0000", 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.385e-06 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829012+0000", 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829015+0000", 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838853+0000", 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098377169999999993 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838915+0000", 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2330999999999997e-05 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838977+0000", 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2160000000000001e-05 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15057 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 757760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10349934099999999, 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010775811, 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15057 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829022+0000", 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3779999999999999e-06 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829028+0000", 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8629999999999996e-06 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829031+0000", 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3069999999999998e-06 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839646+0000", 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010614887999999999 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839722+0000", 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5773999999999997e-05 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839795+0000", 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3277999999999999e-05 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15070 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 630784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103483824, 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010085462999999999, 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15070 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:13:34.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829041+0000", 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7429999999999999e-06 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829046+0000", 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8449999999999999e-06 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829062+0000", 2024-09-16T11:13:34.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5126000000000001e-05 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838999+0000", 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099370090000000001 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839059+0000", 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0340000000000002e-05 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839120+0000", 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1057999999999997e-05 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15071 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829066+0000", 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10345244100000001, 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010206534, 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15071 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829066+0000", 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829066+0000", 2024-09-16T11:13:34.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829067+0000", 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3600000000000003e-07 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829069+0000", 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.092e-06 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829075+0000", 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829078+0000", 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1149999999999998e-06 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839141+0000", 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01006308 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839202+0000", 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1314000000000003e-05 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839272+0000", 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0234999999999994e-05 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15072 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829081+0000", 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103437031, 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01036364, 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15072 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829081+0000", 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829081+0000", 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829082+0000", 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829087+0000", 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9930000000000003e-06 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829093+0000", 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0109999999999997e-06 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829096+0000", 2024-09-16T11:13:34.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839295+0000", 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010199054000000001 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839374+0000", 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8653999999999996e-05 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839445+0000", 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1373999999999998e-05 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15073 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 757760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829099+0000", 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103419076, 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010513712999999999, 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15073 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829099+0000", 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829099+0000", 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829100+0000", 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0100000000000005e-07 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829102+0000", 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829107+0000", 2024-09-16T11:13:34.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8119999999999996e-06 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829111+0000", 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.151e-06 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839467+0000", 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010356816 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839544+0000", 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6336999999999998e-05 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839613+0000", 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9333999999999997e-05 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15082 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 638976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10340438, 2024-09-16T11:13:34.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010863694, 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15082 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829116+0000", 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499999999999999e-06 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829123+0000", 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1290000000000001e-06 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829126+0000", 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839825+0000", 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010699078000000001 2024-09-16T11:13:34.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839907+0000", 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1600000000000005e-05 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839978+0000", 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1160000000000003e-05 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15083 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103388574, 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011036097999999999, 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15083 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.72e-07 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829132+0000", 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.829e-06 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829138+0000", 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7320000000000003e-06 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829141+0000", 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6729999999999998e-06 2024-09-16T11:13:34.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840006+0000", 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010864821 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840090+0000", 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4274e-05 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840166+0000", 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5396999999999997e-05 2024-09-16T11:13:34.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15084 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829145+0000", 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103373275, 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011218195, 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15084 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829145+0000", 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829145+0000", 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829146+0000", 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829148+0000", 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2139999999999999e-06 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829154+0000", 2024-09-16T11:13:34.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.297e-06 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829158+0000", 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.427e-06 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840195+0000", 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011037748999999999 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840276+0000", 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0778999999999993e-05 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840363+0000", 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7128999999999999e-05 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15085 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 765952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829161+0000", 2024-09-16T11:13:34.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10335710100000001, 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011390020000000001, 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15085 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829161+0000", 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829161+0000", 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829162+0000", 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:13:34.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829163+0000", 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829169+0000", 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1249999999999998e-06 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829173+0000", 2024-09-16T11:13:34.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840393+0000", 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011219998 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840475+0000", 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1871000000000003e-05 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:34.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840551+0000", 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6626999999999997e-05 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15110 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10334181100000001, 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011527979000000001, 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:34.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15110 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:34.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829185+0000", 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2349999999999999e-06 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829191+0000", 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1650000000000003e-06 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829195+0000", 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3680000000000002e-06 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840580+0000", 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011385397 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:34.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840642+0000", 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1808000000000002e-05 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840705+0000", 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2654e-05 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15111 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103319551, 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011656168999999999, 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15111 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-16T11:13:35.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829201+0000", 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8959999999999999e-06 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829207+0000", 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9190000000000004e-06 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829215+0000", 2024-09-16T11:13:35.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5859999999999999e-06 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840730+0000", 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011515413 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840794+0000", 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3643e-05 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840855+0000", 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1299999999999999e-05 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15112 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103299603, 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012718281999999999, 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15112 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829221+0000", 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0219999999999999e-06 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829228+0000", 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829231+0000", 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5949999999999999e-06 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840880+0000", 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011648481 2024-09-16T11:13:35.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840941+0000", 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1258000000000002e-05 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841937+0000", 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099617799999999991 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15113 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 761856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103282788, 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011970173000000001, 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15113 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829238+0000", 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1059999999999998e-06 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829247+0000", 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0259999999999996e-06 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829251+0000", 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6239999999999999e-06 2024-09-16T11:13:35.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841072+0000", 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011821645 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841138+0000", 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6025999999999994e-05 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841206+0000", 2024-09-16T11:13:35.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7413e-05 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15126 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829257+0000", 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103261212, 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012124447, 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15126 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829257+0000", 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829257+0000", 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829258+0000", 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829263+0000", 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1100000000000002e-06 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829272+0000", 2024-09-16T11:13:35.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2690000000000005e-06 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829275+0000", 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.422e-06 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841236+0000", 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011960507 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841315+0000", 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9387000000000004e-05 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841382+0000", 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6359000000000006e-05 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15127 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 675840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:35.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10323884899999999, 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012285483, 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15127 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829282+0000", 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-16T11:13:35.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829290+0000", 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3939999999999996e-06 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829294+0000", 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.568e-06 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841407+0000", 2024-09-16T11:13:35.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012112902 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841490+0000", 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3541000000000005e-05 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841565+0000", 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4543999999999996e-05 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15128 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.10322034500000001, 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012457789, 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15128 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1699999999999999e-07 2024-09-16T11:13:35.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829301+0000", 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1100000000000001e-06 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829309+0000", 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7679999999999997e-06 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829313+0000", 2024-09-16T11:13:35.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841593+0000", 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012279516000000001 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841674+0000", 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0995000000000006e-05 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841756+0000", 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2237999999999994e-05 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15129 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 761856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.103201424, 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012611799, 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:35.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15129 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:35.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829319+0000", 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.939e-06 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829325+0000", 2024-09-16T11:13:35.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8799999999999996e-06 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829328+0000", 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2389999999999998e-06 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841781+0000", 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012452995999999999 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841855+0000", 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3856999999999994e-05 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841929+0000", 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3516999999999994e-05 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:35.031 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:35.032 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.032 INFO:teuthology.orchestra.run.smithi162.stdout:1214 read 0xde177d thru 0xdebad3 (0xa357 bytes) 2024-09-16T11:13:35.033 INFO:teuthology.orchestra.run.smithi162.stdout:1215 trunc from 0xe46076b to 0xa200441 2024-09-16T11:13:35.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:1217 writesame 0x866f339 thru 0x867002e data_size 0x7(0xcf6 bytes) 2024-09-16T11:13:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:1219 write 0x6e8b0ab thru 0x6e92d42 (0x7c98 bytes) 2024-09-16T11:13:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:1223 read 0x193d4e5 thru 0x194631b (0x8e37 bytes) 2024-09-16T11:13:35.035 INFO:teuthology.orchestra.run.smithi162.stdout:1224 write 0xdbc16b2 thru 0xdbcbcd0 (0xa61f bytes) 2024-09-16T11:13:35.035 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.035 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.035 INFO:teuthology.orchestra.run.smithi162.stdout:1227 trunc from 0xdbcbcd1 to 0xe174756 2024-09-16T11:13:35.035 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:35.029+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:35.039 INFO:teuthology.orchestra.run.smithi162.stdout:1229 writesame 0x6ab6b94 thru 0x6abce63 data_size 0x22(0x62d0 bytes) 2024-09-16T11:13:35.047 INFO:teuthology.orchestra.run.smithi162.stdout:1230 write 0x2495fe7 thru 0x24a374f (0xd769 bytes) 2024-09-16T11:13:35.057 INFO:teuthology.orchestra.run.smithi162.stdout:1231 punch from 0x444f944 to 0x445d710, (0xddcc bytes) 2024-09-16T11:13:35.057 INFO:teuthology.orchestra.run.smithi162.stdout:1232 read 0x126251b thru 0x126d770 (0xb256 bytes) 2024-09-16T11:13:35.060 INFO:teuthology.orchestra.run.smithi162.stdout:1233 write 0x93fc19f thru 0x93fd699 (0x14fb bytes) 2024-09-16T11:13:35.072 INFO:teuthology.orchestra.run.smithi162.stdout:1234 writesame 0xc4c016 thru 0xc59b4c data_size 0x1(0xdb37 bytes) 2024-09-16T11:13:35.126 INFO:teuthology.orchestra.run.smithi162.stdout:1235 write 0x7a277ee thru 0x7a350f6 (0xd909 bytes) 2024-09-16T11:13:35.133 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:35.129+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:35.134 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.134 INFO:teuthology.orchestra.run.smithi162.stdout:1241 punch from 0x59db302 to 0x59e0d14, (0x5a12 bytes) 2024-09-16T11:13:35.146 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:35.147 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:35.147 INFO:teuthology.orchestra.run.smithi162.stdout:1243 trunc from 0xe174756 to 0x7bd763d 2024-09-16T11:13:35.157 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:35.171 INFO:teuthology.orchestra.run.smithi162.stdout:1244 read 0x3345890 thru 0x3349b7e (0x42ef bytes) 2024-09-16T11:13:35.172 INFO:teuthology.orchestra.run.smithi162.stdout:1245 write 0x153cdda thru 0x154983e (0xca65 bytes) 2024-09-16T11:13:35.186 INFO:teuthology.orchestra.run.smithi162.stdout:1246 write 0x852c30 thru 0x856ef3 (0x42c4 bytes) 2024-09-16T11:13:35.193 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.193 INFO:teuthology.orchestra.run.smithi162.stdout:1252 read 0x9ea6cc thru 0x9f4fc2 (0xa8f7 bytes) 2024-09-16T11:13:35.195 INFO:teuthology.orchestra.run.smithi162.stdout:1253 writesame 0x2d2a2ed thru 0x2d39c1c data_size 0x90(0xf930 bytes) 2024-09-16T11:13:35.200 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:35.214 INFO:teuthology.orchestra.run.smithi162.stdout:1254 read 0x1f77fc1 thru 0x1f81be5 (0x9c25 bytes) 2024-09-16T11:13:35.214 INFO:teuthology.orchestra.run.smithi162.stdout:1255 punch from 0x5f0fb13 to 0x5f1db4f, (0xe03c bytes) 2024-09-16T11:13:35.214 INFO:teuthology.orchestra.run.smithi162.stdout:1256 punch from 0x75b9dda to 0x75bb56b, (0x1791 bytes) 2024-09-16T11:13:35.215 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.215 INFO:teuthology.orchestra.run.smithi162.stdout:1258 read 0x329f216 thru 0x32a995b (0xa746 bytes) 2024-09-16T11:13:35.216 INFO:teuthology.orchestra.run.smithi162.stdout:1259 write 0x2744eb5 thru 0x2745f7e (0x10ca bytes) 2024-09-16T11:13:35.226 INFO:teuthology.orchestra.run.smithi162.stdout:1260 writesame 0x2c8b1a9 thru 0x2c99d80 data_size 0x4(0xebd8 bytes) 2024-09-16T11:13:35.233 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:35.229+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:35.240 INFO:teuthology.orchestra.run.smithi162.stdout:1261 punch from 0x2a3733d to 0x2a44aeb, (0xd7ae bytes) 2024-09-16T11:13:35.254 INFO:teuthology.orchestra.run.smithi162.stdout:1263 write 0xa4f118a thru 0xa4f7d8f (0x6c06 bytes) 2024-09-16T11:13:35.261 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:35.261 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:35.262 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-16T11:13:35.264 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:13:35.264 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:13:35.265 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-16T11:13:35.267 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:35.268 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:35.268 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:35.277 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:35.278 INFO:teuthology.orchestra.run.smithi162.stdout:1265 writesame 0xd110571 thru 0xd111041 data_size 0xd(0xad1 bytes) 2024-09-16T11:13:35.278 INFO:teuthology.orchestra.run.smithi162.stdout:1266 writesame 0x32e2235 thru 0x32e534b data_size 0x3(0x3117 bytes) 2024-09-16T11:13:35.288 INFO:teuthology.orchestra.run.smithi162.stdout:1268 write 0x3380e09 thru 0x338af0c (0xa104 bytes) 2024-09-16T11:13:35.295 INFO:teuthology.orchestra.run.smithi162.stdout:1270 read 0xd06b5c5 thru 0xd07303f (0x7a7b bytes) 2024-09-16T11:13:35.295 INFO:teuthology.orchestra.run.smithi162.stdout:1271 write 0x836ff0f thru 0x837bad9 (0xbbcb bytes) 2024-09-16T11:13:35.301 INFO:teuthology.orchestra.run.smithi162.stdout:1272 read 0x4480368 thru 0x448d1dc (0xce75 bytes) 2024-09-16T11:13:35.302 INFO:teuthology.orchestra.run.smithi162.stdout:1273 punch from 0x6d2dcda to 0x6d3d7bb, (0xfae1 bytes) 2024-09-16T11:13:35.302 INFO:teuthology.orchestra.run.smithi162.stdout:1274 read 0xb8c1258 thru 0xb8c2c15 (0x19be bytes) 2024-09-16T11:13:35.302 INFO:teuthology.orchestra.run.smithi162.stdout:1276 write 0x9ef8ae thru 0x9f723e (0x7991 bytes) 2024-09-16T11:13:35.316 INFO:teuthology.orchestra.run.smithi162.stdout:1277 trunc from 0xd111042 to 0x5b06026 2024-09-16T11:13:35.334 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:35.329+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:35.336 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:35.351 INFO:teuthology.orchestra.run.smithi162.stdout:1279 trunc from 0x5b06026 to 0x5fff6e4 2024-09-16T11:13:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:1280 read 0x3c4f1df thru 0x3c50fd9 (0x1dfb bytes) 2024-09-16T11:13:35.359 INFO:teuthology.orchestra.run.smithi162.stdout:1281 write 0x266d28b thru 0x2675dc2 (0x8b38 bytes) 2024-09-16T11:13:35.374 INFO:teuthology.orchestra.run.smithi162.stdout:1282 trunc from 0x5fff6e4 to 0x2b7782e 2024-09-16T11:13:35.375 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:35.375 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:13:35.375 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:35.382 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:35.400 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:35.400 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:35.411 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:35.412 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:35.416 INFO:teuthology.orchestra.run.smithi162.stdout:1284 trunc from 0x2b7782e to 0x2d4a40c 2024-09-16T11:13:35.431 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:35.436 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:35.433+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:35.440 INFO:teuthology.orchestra.run.smithi162.stdout:1285 write 0x61c8f47 thru 0x61cbe44 (0x2efe bytes) 2024-09-16T11:13:35.454 INFO:teuthology.orchestra.run.smithi162.stdout:1286 trunc from 0x61cbe45 to 0x144a82b 2024-09-16T11:13:35.461 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:35.486 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:35.490 INFO:teuthology.orchestra.run.smithi162.stdout:1287 punch from 0x97f203 to 0x98ea34, (0xf831 bytes) 2024-09-16T11:13:35.510 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:35.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:35.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:35.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:35.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:35.512 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:35.513 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:35.513 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:35.513 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:35.520 INFO:teuthology.orchestra.run.smithi162.stdout:1289 write 0xd919739 thru 0xd925502 (0xbdca bytes) 2024-09-16T11:13:35.521 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:35.521 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:35.528 INFO:teuthology.orchestra.run.smithi162.stdout:1290 write 0x91ff34 thru 0x92826e (0x833b bytes) 2024-09-16T11:13:35.531 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:35.536 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:35.533+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:35.539 INFO:teuthology.orchestra.run.smithi162.stdout:1291 read 0x53d0287 thru 0x53d999b (0x9715 bytes) 2024-09-16T11:13:35.539 INFO:teuthology.orchestra.run.smithi162.stdout:1292 read 0x34476ae thru 0x3449a04 (0x2357 bytes) 2024-09-16T11:13:35.549 INFO:teuthology.orchestra.run.smithi162.stdout:1293 writesame 0xeb4075c thru 0xeb4fa52 data_size 0x3(0xf2f7 bytes) 2024-09-16T11:13:35.549 INFO:teuthology.orchestra.run.smithi162.stdout:1294 read 0x82fd89e thru 0x830505e (0x77c1 bytes) 2024-09-16T11:13:35.567 INFO:teuthology.orchestra.run.smithi162.stdout:1295 trunc from 0xeb4fa53 to 0x5047c5c 2024-09-16T11:13:35.586 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:35.607 INFO:teuthology.orchestra.run.smithi162.stdout:1297 write 0x900da96 thru 0x900db92 (0xfd bytes) 2024-09-16T11:13:35.611 INFO:teuthology.orchestra.run.smithi162.stdout:1298 write 0x7a01048 thru 0x7a0abb7 (0x9b70 bytes) 2024-09-16T11:13:35.617 INFO:teuthology.orchestra.run.smithi162.stdout:1299 read 0x1d45eb7 thru 0x1d49cac (0x3df6 bytes) 2024-09-16T11:13:35.618 INFO:teuthology.orchestra.run.smithi162.stdout:1300 write 0xd1e265 thru 0xd22b89 (0x4925 bytes) 2024-09-16T11:13:35.637 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:35.633+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:35.641 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:35.641 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:35.642 INFO:teuthology.orchestra.run.smithi162.stdout:1301 trunc from 0x900db93 to 0x2120095 2024-09-16T11:13:35.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:35.658 INFO:teuthology.orchestra.run.smithi162.stdout:1302 clone 8 order 18 su 8192 sc 3 2024-09-16T11:13:35.738 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:35.733+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:35.752 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:35.753 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:35.761 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:35.839 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:35.837+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:35.859 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:35.859 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:35.868 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:35.940 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:35.933+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:35.965 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:35.965 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:35.975 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:36.041 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:36.037+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:36.121 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.185 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.187 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:36.188 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:36.185+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:36.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:13:36.288 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:36.285+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:36.298 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.298 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.308 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:36.364 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone7 from 0x2120095 (overlap 0x144a82b) to 0x0 2024-09-16T11:13:36.389 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:36.385+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:36.424 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.424 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.434 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:36.489 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:36.485+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:36.552 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.552 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.563 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:36.611 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:36.609+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:36.650 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T11:13:36.692 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:36.689+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:36.716 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.716 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.725 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:36.792 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:36.785+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:36.825 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.826 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.834 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:36.893 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:36.889+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:36.933 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:36.933 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:36.942 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:36.994 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:36.989+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:37.041 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:37.041 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:37.050 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:37.095 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:37.089+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:37.097 INFO:teuthology.orchestra.run.smithi162.stdout:1305 trunc from 0x2120095 to 0x4961fc6 2024-09-16T11:13:37.115 INFO:teuthology.orchestra.run.smithi162.stdout:1306 writesame 0x2075da7 thru 0x208186d data_size 0x49(0xbac7 bytes) 2024-09-16T11:13:37.125 INFO:teuthology.orchestra.run.smithi162.stdout:1307 read 0x31a5f76 thru 0x31aa18b (0x4216 bytes) 2024-09-16T11:13:37.125 INFO:teuthology.orchestra.run.smithi162.stdout:1309 punch from 0x3b1809 to 0x3bd525, (0xbd1c bytes) 2024-09-16T11:13:37.125 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:37.151 INFO:teuthology.orchestra.run.smithi162.stdout:1311 write 0xc4aedef thru 0xc4b36df (0x48f1 bytes) 2024-09-16T11:13:37.160 INFO:teuthology.orchestra.run.smithi162.stdout:1312 write 0xafb8807 thru 0xafc2186 (0x9980 bytes) 2024-09-16T11:13:37.168 INFO:teuthology.orchestra.run.smithi162.stdout:1313 writesame 0xaf9bb70 thru 0xafa1287 data_size 0xc(0x5718 bytes) 2024-09-16T11:13:37.173 INFO:teuthology.orchestra.run.smithi162.stdout:1315 punch from 0x7d9646b to 0x7da0730, (0xa2c5 bytes) 2024-09-16T11:13:37.173 INFO:teuthology.orchestra.run.smithi162.stdout:1317 read 0x90366e thru 0x90c14b (0x8ade bytes) 2024-09-16T11:13:37.183 INFO:teuthology.orchestra.run.smithi162.stdout:1318 trunc from 0xc4b36e0 to 0x88dfd26 2024-09-16T11:13:37.195 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:37.193+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:37.199 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:37.199 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:37.209 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:37.215 INFO:teuthology.orchestra.run.smithi162.stdout:1319 writesame 0x994c119 thru 0x995075e data_size 0x23(0x4646 bytes) 2024-09-16T11:13:37.215 INFO:teuthology.orchestra.run.smithi162.stdout:1320 writesame 0x3b33a72 thru 0x3b3eefb data_size 0x3(0xb48a bytes) 2024-09-16T11:13:37.244 INFO:teuthology.orchestra.run.smithi162.stdout:1322 trunc from 0x995075f to 0x5743862 2024-09-16T11:13:37.259 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:13:37.284 INFO:teuthology.orchestra.run.smithi162.stdout:1323 writesame 0xe84803b thru 0xe84d737 data_size 0xd(0x56fd bytes) 2024-09-16T11:13:37.285 INFO:teuthology.orchestra.run.smithi162.stdout:1324 read 0x1644854 thru 0x1649d5f (0x550c bytes) 2024-09-16T11:13:37.285 INFO:teuthology.orchestra.run.smithi162.stdout:1325 read 0xe511635 thru 0xe51db0d (0xc4d9 bytes) 2024-09-16T11:13:37.285 INFO:teuthology.orchestra.run.smithi162.stdout:1326 clone 9 order 25 su 1048576 sc 3 2024-09-16T11:13:37.296 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:37.289+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:37.306 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:37.307 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:37.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:37.397 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:37.393+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:37.413 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:37.413 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:37.423 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:37.498 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:37.493+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:37.523 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:37.569 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:37.569 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:37.578 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:37.601 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:37.597+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:37.667 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:37.677 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:37.677 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:37.687 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:37.688 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:37.702 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:37.697+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:37.710 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:37.731 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:37.753 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:37.773 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:37.792 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:37.792 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:37.792 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:37.792 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:37.796 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:37.801 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:37.803 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:37.797+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:37.819 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:37.821 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:37.822 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:37.898 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:37.899 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:37.903 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:37.901+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:37.908 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:38.004 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:37.997+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:38.010 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:756 4.b 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:38.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.221701243999998, 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.035717454000000003, 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 756 2024-09-16T11:13:38.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784868+0000", 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.784871+0000", 2024-09-16T11:13:38.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.497e-06 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790900+0000", 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0060292039999999998 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790939+0000", 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8844999999999998e-05 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.790951+0000", 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2143999999999999e-05 2024-09-16T11:13:38.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791017+0000", 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5969999999999993e-05 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791210+0000", 2024-09-16T11:13:38.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000192993 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.791635+0000", 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042530200000000002 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.812994+0000", 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021359210999999999 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820448+0000", 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0074540709999999996 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820509+0000", 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0625999999999999e-05 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820586+0000", 2024-09-16T11:13:38.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6590999999999999e-05 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:2730 4.3 4:c29a4800:::rbd_object_map.11b5220ac4a2:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:25.402039+0000", 2024-09-16T11:13:38.021 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.604530607000001, 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.013361654000000001, 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 2730 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.022 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402039+0000", 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402039+0000", 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402040+0000", 2024-09-16T11:13:38.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5099999999999998e-07 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402043+0000", 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.286e-06 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402055+0000", 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.17e-05 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.402060+0000", 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9309999999999999e-06 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.407941+0000", 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0058809550000000002 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.408095+0000", 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015458 2024-09-16T11:13:38.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.408767+0000", 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00067182899999999998 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.410676+0000", 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00190877 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.415278+0000", 2024-09-16T11:13:38.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004602755 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.415344+0000", 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5782000000000001e-05 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:25.415400+0000", 2024-09-16T11:13:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6215000000000003e-05 2024-09-16T11:13:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15056 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 716800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177566364, 2024-09-16T11:13:38.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099743899999999996, 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15056 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829003+0000", 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829006+0000", 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.385e-06 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829012+0000", 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9039999999999997e-06 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829015+0000", 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5020000000000001e-06 2024-09-16T11:13:38.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838853+0000", 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0098377169999999993 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838915+0000", 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2330999999999997e-05 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838977+0000", 2024-09-16T11:13:38.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2160000000000001e-05 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15057 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 757760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177550423, 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010775811, 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.034 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15057 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829019+0000", 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829022+0000", 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3779999999999999e-06 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829028+0000", 2024-09-16T11:13:38.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8629999999999996e-06 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829031+0000", 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3069999999999998e-06 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839646+0000", 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010614887999999999 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839722+0000", 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5773999999999997e-05 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839795+0000", 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3277999999999999e-05 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15070 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 630784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.038 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177534906, 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010085462999999999, 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15070 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829035+0000", 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829041+0000", 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7429999999999999e-06 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829046+0000", 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8449999999999999e-06 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829062+0000", 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5126000000000001e-05 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.838999+0000", 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0099370090000000001 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839059+0000", 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0340000000000002e-05 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839120+0000", 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1057999999999997e-05 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15071 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 671744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829066+0000", 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1775035229999999, 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010206534, 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15071 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829066+0000", 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829066+0000", 2024-09-16T11:13:38.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829067+0000", 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3600000000000003e-07 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829069+0000", 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.092e-06 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829075+0000", 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0619999999999997e-06 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829078+0000", 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1149999999999998e-06 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839141+0000", 2024-09-16T11:13:38.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01006308 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839202+0000", 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1314000000000003e-05 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839272+0000", 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0234999999999994e-05 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15072 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 712704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829081+0000", 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1774881129999999, 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01036364, 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15072 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829081+0000", 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829081+0000", 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829082+0000", 2024-09-16T11:13:38.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829087+0000", 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9930000000000003e-06 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829093+0000", 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0109999999999997e-06 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829096+0000", 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839295+0000", 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010199054000000001 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839374+0000", 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8653999999999996e-05 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839445+0000", 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1373999999999998e-05 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15073 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 757760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829099+0000", 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1774701580000002, 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010513712999999999, 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15073 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829099+0000", 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829099+0000", 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829100+0000", 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0100000000000005e-07 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829102+0000", 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6619999999999999e-06 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829107+0000", 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8119999999999996e-06 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829111+0000", 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.151e-06 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839467+0000", 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010356816 2024-09-16T11:13:38.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839544+0000", 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6336999999999998e-05 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839613+0000", 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9333999999999997e-05 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15082 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 638976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1774554620000002, 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010863694, 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15082 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829114+0000", 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829116+0000", 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499999999999999e-06 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829123+0000", 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1290000000000001e-06 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829126+0000", 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:13:38.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839825+0000", 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010699078000000001 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839907+0000", 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1600000000000005e-05 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.839978+0000", 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1160000000000003e-05 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15083 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1774396559999998, 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011036097999999999, 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.061 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15083 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:38.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829130+0000", 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.72e-07 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829132+0000", 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.829e-06 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829138+0000", 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7320000000000003e-06 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829141+0000", 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6729999999999998e-06 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840006+0000", 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.010864821 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840090+0000", 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4274e-05 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840166+0000", 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5396999999999997e-05 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.065 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15084 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829145+0000", 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177424357, 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011218195, 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15084 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829145+0000", 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829145+0000", 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829146+0000", 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829148+0000", 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2139999999999999e-06 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829154+0000", 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.297e-06 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829158+0000", 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.427e-06 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840195+0000", 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011037748999999999 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840276+0000", 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0778999999999993e-05 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840363+0000", 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7128999999999999e-05 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.069 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15085 4.1e 4:79a9b8ac:::rbd_data.11b52f8117af.000000000000009c:a [read 765952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829161+0000", 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1774081829999998, 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011390020000000001, 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15085 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829161+0000", 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829161+0000", 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829162+0000", 2024-09-16T11:13:38.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3000000000000002e-07 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829163+0000", 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.73e-06 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829169+0000", 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1249999999999998e-06 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829173+0000", 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840393+0000", 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011219998 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840475+0000", 2024-09-16T11:13:38.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1871000000000003e-05 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840551+0000", 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6626999999999997e-05 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15110 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1773928929999999, 2024-09-16T11:13:38.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011527979000000001, 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15110 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829177+0000", 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829185+0000", 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2349999999999999e-06 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829191+0000", 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1650000000000003e-06 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829195+0000", 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3680000000000002e-06 2024-09-16T11:13:38.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840580+0000", 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011385397 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840642+0000", 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1808000000000002e-05 2024-09-16T11:13:38.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840705+0000", 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2654e-05 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15111 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 679936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1773706329999998, 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011656168999999999, 2024-09-16T11:13:38.079 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15111 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829199+0000", 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1199999999999998e-07 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829201+0000", 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8959999999999999e-06 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829207+0000", 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9190000000000004e-06 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829215+0000", 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5859999999999999e-06 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840730+0000", 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011515413 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840794+0000", 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3643e-05 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840855+0000", 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1299999999999999e-05 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15112 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177350685, 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012718281999999999, 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15112 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829219+0000", 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829221+0000", 2024-09-16T11:13:38.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0219999999999999e-06 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829228+0000", 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3559999999999998e-06 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829231+0000", 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5949999999999999e-06 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840880+0000", 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011648481 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.840941+0000", 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1258000000000002e-05 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841937+0000", 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00099617799999999991 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15113 4.1e 4:7803e381:::rbd_data.11b52f8117af.00000000000000a3:a [read 761856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.17733387, 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011970173000000001, 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15113 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:38.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829236+0000", 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999998e-07 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829238+0000", 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1059999999999998e-06 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829247+0000", 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0259999999999996e-06 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829251+0000", 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6239999999999999e-06 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841072+0000", 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011821645 2024-09-16T11:13:38.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841138+0000", 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6025999999999994e-05 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841206+0000", 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7413e-05 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15126 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 634880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829257+0000", 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177312294, 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012124447, 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15126 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829257+0000", 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829257+0000", 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829258+0000", 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:38.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829263+0000", 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1100000000000002e-06 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829272+0000", 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2690000000000005e-06 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829275+0000", 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.422e-06 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841236+0000", 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011960507 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841315+0000", 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9387000000000004e-05 2024-09-16T11:13:38.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841382+0000", 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6359000000000006e-05 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15127 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 675840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.097 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1772899309999998, 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012285483, 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15127 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829280+0000", 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0800000000000004e-07 2024-09-16T11:13:38.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829282+0000", 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9259999999999999e-06 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829290+0000", 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3939999999999996e-06 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829294+0000", 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.568e-06 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841407+0000", 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012112902 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841490+0000", 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3541000000000005e-05 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841565+0000", 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4543999999999996e-05 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15128 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:38.102 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.177271427, 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012457789, 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15128 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829298+0000", 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1699999999999999e-07 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829301+0000", 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1100000000000001e-06 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829309+0000", 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7679999999999997e-06 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829313+0000", 2024-09-16T11:13:38.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841593+0000", 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012279516000000001 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841674+0000", 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0995000000000006e-05 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841756+0000", 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2237999999999994e-05 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:15129 4.16 4:6ad7bd16:::rbd_data.11b52f8117af.00000000000000a7:a [read 761856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1772525059999999, 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012611799, 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 15129 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829317+0000", 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7099999999999997e-07 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829319+0000", 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.939e-06 2024-09-16T11:13:38.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829325+0000", 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8799999999999996e-06 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.829328+0000", 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2389999999999998e-06 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841781+0000", 2024-09-16T11:13:38.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.012452995999999999 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841855+0000", 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3856999999999994e-05 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.841929+0000", 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3516999999999994e-05 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.112 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.113 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:38.113 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:38.109+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:38.214 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:38.209+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:38.220 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.220 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:38.220 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:38.220 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.230 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:38.315 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:38.313+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:38.342 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.342 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:38.342 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:38.343 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:38.343 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.353 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:38.395 INFO:teuthology.orchestra.run.smithi162.stdout:flattening image image_client.0-clone8 2024-09-16T11:13:38.416 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:38.413+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:38.489 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:756 4.b e39/31 4:d364d01a:::rbd_data.11b585795d6b.0000000000000004:head v 39'5, mlcod=38'2)", 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.693481133999999, 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027783187000000001, 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792383+0000", 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.792386+0000", 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.667e-06 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799237+0000", 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.006850737 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799272+0000", 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5147000000000001e-05 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799284+0000", 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2391999999999999e-05 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799389+0000", 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000104759 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.799444+0000", 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5118000000000003e-05 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.819973+0000", 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020528497 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.820167+0000", 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019387000000000001 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:828 4.7 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13441105~9082] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 17.628665388999998, 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042301801999999999, 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 828 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857199+0000", 2024-09-16T11:13:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.857203+0000", 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7299999999999999e-06 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862148+0000", 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0049449400000000001 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862190+0000", 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2153000000000001e-05 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862203+0000", 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2604e-05 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862330+0000", 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012740900000000001 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.862670+0000", 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00033967399999999999 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 5", 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.863645+0000", 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00097586600000000004 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.884164+0000", 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020518723999999999 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899338+0000", 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.015173623000000001 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899435+0000", 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7671999999999996e-05 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:20.899501+0000", 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5407000000000006e-05 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:3822 4.4 4:2775f313:::rbd_data.11b5b93336d2.0000000000000017:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.427550+0000", 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.058314340000001, 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023090313000000001, 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3822 2024-09-16T11:13:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427550+0000", 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427550+0000", 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427553+0000", 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8760000000000001e-06 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427610+0000", 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7056999999999999e-05 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427631+0000", 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0954999999999998e-05 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427639+0000", 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8029999999999997e-06 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427707+0000", 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8553000000000006e-05 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.427852+0000", 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014468400000000001 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.433095+0000", 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052433460000000003 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.450343+0000", 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017247574000000002 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.450374+0000", 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1353999999999998e-05 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.450520+0000", 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014562200000000001 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.450640+0000", 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120489 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23569 4.4 4:26f59c9d:::rbd_data.11b52f8117af.0000000000000004:a [read 499712~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463267+0000", 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022597526, 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020907465, 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23569 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463267+0000", 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463267+0000", 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463267+0000", 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.39e-07 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463269+0000", 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.809e-06 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463274+0000", 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2669999999999997e-06 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463277+0000", 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2469999999999998e-06 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.481497+0000", 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018220407000000001 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.481758+0000", 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260676 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484174+0000", 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024167199999999998 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23580 4.4 4:26f59c9d:::rbd_data.11b52f8117af.0000000000000004:a [read 466944~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463287+0000", 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022577317, 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018866923000000001, 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23580 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463287+0000", 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463287+0000", 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463287+0000", 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463289+0000", 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.299e-06 2024-09-16T11:13:38.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463292+0000", 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7949999999999999e-06 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463294+0000", 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1019999999999999e-06 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482013+0000", 2024-09-16T11:13:38.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018719514999999999 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482076+0000", 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2708999999999999e-05 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482154+0000", 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8246999999999998e-05 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23585 4.1c 4:3b3c93cf:::rbd_data.11b52f8117af.0000000000000009:a [read 471040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463295+0000", 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022569077, 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019016354999999999, 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23585 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463295+0000", 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463295+0000", 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463296+0000", 2024-09-16T11:13:38.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463297+0000", 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3039999999999999e-06 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463300+0000", 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8710000000000001e-06 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463302+0000", 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3e-06 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482176+0000", 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01887395 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482239+0000", 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3267000000000003e-05 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482312+0000", 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2410999999999995e-05 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23590 4.4 4:21db1524:::rbd_data.11b52f8117af.0000000000000006:a [read 475136~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463304+0000", 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022560555999999999, 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019160785999999999, 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23590 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463304+0000", 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463304+0000", 2024-09-16T11:13:38.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463304+0000", 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4400000000000001e-07 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463305+0000", 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.06e-06 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463308+0000", 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9440000000000001e-06 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463310+0000", 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0080000000000001e-06 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482333+0000", 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01902266 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482396+0000", 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3099e-05 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482465+0000", 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8770999999999996e-05 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23593 4.4 4:26f59c9d:::rbd_data.11b52f8117af.0000000000000004:a [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463312+0000", 2024-09-16T11:13:38.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022552395999999999, 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019308394, 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23593 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463312+0000", 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463312+0000", 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463312+0000", 2024-09-16T11:13:38.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463317+0000", 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7570000000000004e-06 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463321+0000", 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4249999999999998e-06 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463323+0000", 2024-09-16T11:13:38.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482486+0000", 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019163515999999998 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482549+0000", 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2770999999999999e-05 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482620+0000", 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1673000000000003e-05 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23599 4.1c 4:3b3c93cf:::rbd_data.11b52f8117af.0000000000000009:a [read 483328~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463325+0000", 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022539722000000002, 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019448323, 2024-09-16T11:13:38.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23599 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463325+0000", 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463325+0000", 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463325+0000", 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463326+0000", 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4440000000000001e-06 2024-09-16T11:13:38.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463330+0000", 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7859999999999998e-06 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463332+0000", 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482641+0000", 2024-09-16T11:13:38.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019308241 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482702+0000", 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1179999999999994e-05 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482773+0000", 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1159000000000001e-05 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23602 4.4 4:21db1524:::rbd_data.11b52f8117af.0000000000000006:a [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463334+0000", 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022530115999999999, 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019594248000000002, 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23602 2024-09-16T11:13:38.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463334+0000", 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463334+0000", 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463335+0000", 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6899999999999999e-07 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463336+0000", 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2500000000000001e-06 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463339+0000", 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9809999999999999e-06 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463341+0000", 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2249999999999999e-06 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482794+0000", 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019453137999999998 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482857+0000", 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2817999999999994e-05 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482929+0000", 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1567000000000001e-05 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23608 4.4 4:21db1524:::rbd_data.11b52f8117af.0000000000000006:a [read 450560~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463343+0000", 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022521572, 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019740165, 2024-09-16T11:13:38.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23608 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463343+0000", 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463343+0000", 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463343+0000", 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463344+0000", 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.243e-06 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463347+0000", 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9340000000000002e-06 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463350+0000", 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1900000000000002e-06 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.482949+0000", 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019599351000000001 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483012+0000", 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3170000000000007e-05 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483083+0000", 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1020999999999996e-05 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23610 4.4 4:26f59c9d:::rbd_data.11b52f8117af.0000000000000004:a [read 491520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463351+0000", 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022513169999999999, 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019885124000000001, 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23610 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463351+0000", 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463351+0000", 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463352+0000", 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6199999999999999e-07 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463353+0000", 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.17e-06 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463356+0000", 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.898e-06 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463358+0000", 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8600000000000001e-06 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483103+0000", 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019744599000000002 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483165+0000", 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2065999999999998e-05 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483236+0000", 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1268999999999998e-05 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23617 4.4 4:26f59c9d:::rbd_data.11b52f8117af.0000000000000004:a [read 454656~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.547 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463360+0000", 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022503975999999998, 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020028247999999998, 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23617 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463360+0000", 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463360+0000", 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463361+0000", 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6199999999999999e-07 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463362+0000", 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1710000000000001e-06 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463365+0000", 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9610000000000001e-06 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463367+0000", 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1459999999999999e-06 2024-09-16T11:13:38.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483257+0000", 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019889876000000001 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483319+0000", 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2444e-05 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483389+0000", 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9387999999999994e-05 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23624 4.1c 4:3b3c93cf:::rbd_data.11b52f8117af.0000000000000009:a [read 495616~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463376+0000", 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022488429000000001, 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020167753, 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23624 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463376+0000", 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463376+0000", 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463376+0000", 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4799999999999999e-07 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463378+0000", 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.906e-06 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463385+0000", 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2380000000000002e-06 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463388+0000", 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3790000000000002e-06 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483410+0000", 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020022422000000002 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483474+0000", 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3745999999999996e-05 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483544+0000", 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9714000000000004e-05 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23631 4.1c 4:3b3c93cf:::rbd_data.11b52f8117af.0000000000000009:a [read 458752~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463423+0000", 2024-09-16T11:13:38.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022441859000000002, 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020276063, 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23631 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463423+0000", 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463423+0000", 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463423+0000", 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9999999999999997e-07 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463426+0000", 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6400000000000001e-06 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463434+0000", 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2719999999999997e-06 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463438+0000", 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2560000000000004e-06 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483564+0000", 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020125764000000001 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483627+0000", 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2841000000000004e-05 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483699+0000", 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1589999999999997e-05 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23632 4.4 4:21db1524:::rbd_data.11b52f8117af.0000000000000006:a [read 499712~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463443+0000", 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022421402999999999, 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020410840999999999, 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23632 2024-09-16T11:13:38.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463443+0000", 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463443+0000", 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463444+0000", 2024-09-16T11:13:38.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3499999999999996e-07 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463446+0000", 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463452+0000", 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3430000000000001e-06 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463456+0000", 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.58e-06 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483720+0000", 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020264098000000001 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483784+0000", 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4125e-05 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483854+0000", 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9880000000000002e-05 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23637 4.4 4:26f59c9d:::rbd_data.11b52f8117af.0000000000000004:a [read 503808~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.463476+0000", 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.022388936000000002, 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020532973999999999, 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23637 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463476+0000", 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463476+0000", 2024-09-16T11:13:38.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463476+0000", 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1600000000000001e-07 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463478+0000", 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1469999999999999e-06 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463486+0000", 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4370000000000001e-06 2024-09-16T11:13:38.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.463490+0000", 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7840000000000003e-06 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483875+0000", 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020384921 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.483939+0000", 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3205000000000002e-05 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484009+0000", 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9963999999999996e-05 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23656 4.1c 4:3b3c93cf:::rbd_data.11b52f8117af.0000000000000009:a [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.465336+0000", 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.020528344, 2024-09-16T11:13:38.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018907222000000001, 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23656 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465336+0000", 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465336+0000", 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465337+0000", 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7300000000000005e-07 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465339+0000", 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.52e-06 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465346+0000", 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7449999999999998e-06 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465350+0000", 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1030000000000004e-06 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484030+0000", 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018679391 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484100+0000", 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484243+0000", 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014298300000000001 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23659 4.4 4:21db1524:::rbd_data.11b52f8117af.0000000000000006:a [read 552960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.573 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.465377+0000", 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.020487839000000001, 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018970830000000001, 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23659 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465377+0000", 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465377+0000", 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465377+0000", 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465379+0000", 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.153e-06 2024-09-16T11:13:38.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465386+0000", 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2489999999999998e-06 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465405+0000", 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8989000000000001e-05 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484200+0000", 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018795835 2024-09-16T11:13:38.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484272+0000", 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2099000000000002e-05 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484347+0000", 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5007999999999997e-05 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.577 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:23665 4.4 4:21db1524:::rbd_data.11b52f8117af.0000000000000006:a [read 516096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:38.465427+0000", 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.020437666, 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019146951999999998, 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 23665 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465427+0000", 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465427+0000", 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465427+0000", 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465430+0000", 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3939999999999999e-06 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465435+0000", 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7799999999999997e-06 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.465439+0000", 2024-09-16T11:13:38.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.579e-06 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484412+0000", 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018972637000000001 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484485+0000", 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3650000000000001e-05 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:38.484574+0000", 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8465000000000001e-05 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.582 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:38.584 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:38.577+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:38.685 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:38.681+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:38.724 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.724 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:38.724 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:38.724 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.734 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:38.786 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:38.781+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:38.809 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-16T11:13:38.828 INFO:teuthology.orchestra.run.smithi162.stdout:1329 writesame 0x2dc2095 thru 0x2dc4ff6 data_size 0x5(0x2f62 bytes) 2024-09-16T11:13:38.830 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.830 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:38.830 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:38.830 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:38.830 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.840 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:38.869 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:38.869 INFO:teuthology.orchestra.run.smithi162.stdout:1331 read 0xc29fd54 thru 0xc2a1287 (0x1534 bytes) 2024-09-16T11:13:38.869 INFO:teuthology.orchestra.run.smithi162.stdout:1333 write 0x554439f thru 0x5550aba (0xc71c bytes) 2024-09-16T11:13:38.887 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:38.881+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:38.891 INFO:teuthology.orchestra.run.smithi162.stdout:1335 write 0xb4e3164 thru 0xb4e414e (0xfeb bytes) 2024-09-16T11:13:38.900 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:38.905 INFO:teuthology.orchestra.run.smithi162.stdout:1338 trunc from 0xe84d738 to 0xc4a6be2 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.106261451000002, 2024-09-16T11:13:38.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:38.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:38.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.815686680999999, 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:38.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:38.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:38.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:38.950 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.807278393000001, 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:38.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:38.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:38.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:38.955 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:38.956 INFO:teuthology.orchestra.run.smithi162.stdout:1341 write 0xe8ed2a1 thru 0xe8f861f (0xb37f bytes) 2024-09-16T11:13:38.956 INFO:teuthology.orchestra.run.smithi162.stdout:1342 read 0xc19578a thru 0xc19cfe6 (0x785d bytes) 2024-09-16T11:13:38.956 INFO:teuthology.orchestra.run.smithi162.stdout:1343 write 0xcf9608e thru 0xcf97957 (0x18ca bytes) 2024-09-16T11:13:38.956 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:38.956 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:38.956 INFO:teuthology.orchestra.run.smithi162.stdout:1350 writesame 0x97e318d thru 0x97f262f data_size 0x1(0xf4a3 bytes) 2024-09-16T11:13:38.957 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:39.005 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:39.001+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:39.006 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:39.007 INFO:teuthology.orchestra.run.smithi162.stdout:1352 clone 10 order 23 su 524288 sc 11 2024-09-16T11:13:39.089 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:39.085+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:39.111 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:39.111 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:39.111 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:39.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:39.190 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:39.185+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:39.217 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:39.217 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:39.218 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:39.218 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:39.218 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:39.226 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:39.275 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:13:39.291 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:39.285+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:39.335 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:39.335 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:39.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:39.335 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:39.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:3875 4.17 4:ebebcad9:::rbd_data.11b5b93336d2.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.826256953, 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02646399, 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3875 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503604+0000", 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:13:39.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503618+0000", 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.375e-05 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503642+0000", 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.357e-05 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503651+0000", 2024-09-16T11:13:39.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1060000000000006e-06 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503714+0000", 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3368999999999996e-05 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503856+0000", 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014167699999999999 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.510013+0000", 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061567940000000002 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529819+0000", 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019806462 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529932+0000", 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112724 2024-09-16T11:13:39.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530010+0000", 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8412999999999996e-05 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530066+0000", 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6004000000000001e-05 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:4158 4.0 4:047b769a:::rbd_data.11b5b93336d2.000000000000001d:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.223388705, 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032637175999999997, 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 4158 2024-09-16T11:13:39.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106472+0000", 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.35e-06 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106479+0000", 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106496+0000", 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106503+0000", 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106568+0000", 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4346999999999999e-05 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106646+0000", 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8904000000000002e-05 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.114604+0000", 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079574510000000008 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.138952+0000", 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024348172000000001 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139028+0000", 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6315000000000004e-05 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139080+0000", 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1450999999999999e-05 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139108+0000", 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7855e-05 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13707 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.347 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7814677420000002, 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023978039999999999, 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13707 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9299999999999997e-07 2024-09-16T11:13:39.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548396+0000", 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548405+0000", 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6130000000000004e-06 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548410+0000", 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3639999999999998e-06 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.565053+0000", 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016642410999999999 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.565199+0000", 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014626400000000001 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.565740+0000", 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054094600000000005 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.568994+0000", 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032539610000000001 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572316+0000", 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033225110000000002 2024-09-16T11:13:39.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572342+0000", 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5338e-05 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572370+0000", 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7962000000000001e-05 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13708 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.548415+0000", 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7814439589999997, 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024336200999999998, 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13708 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548415+0000", 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548415+0000", 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548416+0000", 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0699999999999996e-07 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548420+0000", 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548428+0000", 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0160000000000008e-06 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548433+0000", 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3530000000000002e-06 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.566374+0000", 2024-09-16T11:13:39.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017940626000000001 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.566466+0000", 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1642000000000001e-05 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.566854+0000", 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038811 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.569000+0000", 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021461100000000001 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572655+0000", 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003655265 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572716+0000", 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0887000000000002e-05 2024-09-16T11:13:39.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572752+0000", 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5465e-05 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13709 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.7814206659999998, 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024492159999999999, 2024-09-16T11:13:39.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13709 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8000000000000005e-07 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548449+0000", 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8730000000000002e-06 2024-09-16T11:13:39.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548477+0000", 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7846000000000001e-05 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548498+0000", 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0590000000000001e-05 2024-09-16T11:13:39.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.567398+0000", 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01890004 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.567519+0000", 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120987 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.568010+0000", 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049080499999999997 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.569148+0000", 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011385740000000001 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572809+0000", 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036606920000000001 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572869+0000", 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0239999999999999e-05 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572931+0000", 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1733e-05 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13798 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652642+0000", 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.6772171770000002, 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024192179000000001, 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13798 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.367 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652642+0000", 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652642+0000", 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652643+0000", 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3499999999999996e-07 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652646+0000", 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5750000000000001e-06 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652662+0000", 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5675000000000002e-05 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652666+0000", 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1930000000000003e-06 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.671029+0000", 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018363180999999999 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.671142+0000", 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113064 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.671588+0000", 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044516899999999998 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676658+0000", 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050707720000000003 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676740+0000", 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1626999999999996e-05 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676789+0000", 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9224999999999997e-05 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676834+0000", 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5062999999999999e-05 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13799 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.372 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652698+0000", 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.6771608750000002, 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024592899000000001, 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13799 2024-09-16T11:13:39.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652698+0000", 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652698+0000", 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652699+0000", 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8899999999999998e-07 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652703+0000", 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8419999999999998e-06 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652710+0000", 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6640000000000001e-06 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652714+0000", 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.673010+0000", 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020296209999999999 2024-09-16T11:13:39.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.673088+0000", 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8663000000000002e-05 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.673463+0000", 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037484900000000002 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676670+0000", 2024-09-16T11:13:39.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032069279999999999 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677216+0000", 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054598699999999995 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677258+0000", 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2052000000000003e-05 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677291+0000", 2024-09-16T11:13:39.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3002e-05 2024-09-16T11:13:39.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.382 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13800 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652735+0000", 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.6771241620000001, 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025687231000000001, 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13800 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652735+0000", 2024-09-16T11:13:39.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652735+0000", 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652736+0000", 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-16T11:13:39.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652739+0000", 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.625e-06 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652746+0000", 2024-09-16T11:13:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-16T11:13:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652750+0000", 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9369999999999997e-06 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.674187+0000", 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021437060000000001 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.674287+0000", 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100191 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.674772+0000", 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048459700000000002 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676681+0000", 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019092410000000001 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678229+0000", 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001548694 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678350+0000", 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012061999999999999 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678422+0000", 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2269999999999998e-05 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13801 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652787+0000", 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.6770722569999998, 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025949692, 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13801 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.392 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652787+0000", 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652787+0000", 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652788+0000", 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3200000000000004e-07 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652792+0000", 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9530000000000001e-06 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652798+0000", 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6850000000000004e-06 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652802+0000", 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7570000000000002e-06 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.675553+0000", 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022751141999999999 2024-09-16T11:13:39.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.675663+0000", 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109462 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676011+0000", 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034868900000000001 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677038+0000", 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010261230000000001 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678604+0000", 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001566663 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678688+0000", 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3770999999999995e-05 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678737+0000", 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8714999999999997e-05 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13913 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.793159+0000", 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5366999530000003, 2024-09-16T11:13:39.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023814730999999999, 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13913 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793159+0000", 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793159+0000", 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793160+0000", 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2799999999999995e-07 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793165+0000", 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2900000000000002e-06 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793177+0000", 2024-09-16T11:13:39.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1963999999999999e-05 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793185+0000", 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4499999999999998e-06 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.811958+0000", 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018772825999999999 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.812099+0000", 2024-09-16T11:13:39.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141643 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.812557+0000", 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045780199999999999 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.815840+0000", 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032831570000000001 2024-09-16T11:13:39.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816895+0000", 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001054227 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816937+0000", 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2107000000000002e-05 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816974+0000", 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7437000000000001e-05 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13914 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.793192+0000", 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.5366668610000005, 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023917087, 2024-09-16T11:13:39.405 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13914 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793192+0000", 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793192+0000", 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793193+0000", 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4200000000000005e-07 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793198+0000", 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1599999999999997e-06 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793212+0000", 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.394e-05 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793220+0000", 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9529999999999999e-06 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.813913+0000", 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020693096000000001 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.814043+0000", 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129228 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.814628+0000", 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058514299999999999 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816990+0000", 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023619549999999998 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.817035+0000", 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789000000000001e-05 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.817074+0000", 2024-09-16T11:13:39.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9124e-05 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.817109+0000", 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5856999999999999e-05 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.411 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14022 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915633+0000", 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.4142261700000001, 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023566409999999999, 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14022 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915633+0000", 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915633+0000", 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915634+0000", 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1699999999999997e-07 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915639+0000", 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2220000000000001e-06 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915652+0000", 2024-09-16T11:13:39.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2815e-05 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915660+0000", 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0299999999999994e-06 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.931984+0000", 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016323431999999999 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.932097+0000", 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113096 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.932626+0000", 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052935000000000005 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.937477+0000", 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048505900000000001 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939008+0000", 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015318350000000001 2024-09-16T11:13:39.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939118+0000", 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110041 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939200+0000", 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1081999999999994e-05 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.418 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14023 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915669+0000", 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.4141906139999998, 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023738168, 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14023 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.419 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915669+0000", 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915669+0000", 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915670+0000", 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915675+0000", 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1429999999999997e-06 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915687+0000", 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2367e-05 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915698+0000", 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.933458+0000", 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017759792 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.933514+0000", 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5896000000000002e-05 2024-09-16T11:13:39.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.933789+0000", 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027467799999999999 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939230+0000", 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054411609999999999 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939294+0000", 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3927e-05 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939359+0000", 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5839000000000004e-05 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939407+0000", 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7366e-05 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.424 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14024 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915707+0000", 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.4141527289999996, 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023850314000000001, 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14024 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915707+0000", 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915707+0000", 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915708+0000", 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5999999999999991e-07 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915713+0000", 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915729+0000", 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5265e-05 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915736+0000", 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8480000000000001e-06 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.934224+0000", 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018487665 2024-09-16T11:13:39.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.934302+0000", 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7494e-05 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.934651+0000", 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034901600000000001 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939248+0000", 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045969330000000001 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939496+0000", 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024808700000000003 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939530+0000", 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4532000000000001e-05 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939557+0000", 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6662999999999999e-05 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14025 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915744+0000", 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.4141148079999999, 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023955583999999999, 2024-09-16T11:13:39.431 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14025 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915744+0000", 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915744+0000", 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915745+0000", 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.09e-07 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915750+0000", 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.763e-06 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915762+0000", 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1498000000000001e-05 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915769+0000", 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8739999999999995e-06 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.935598+0000", 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019828124999999999 2024-09-16T11:13:39.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.935733+0000", 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000135809 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.936414+0000", 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068093099999999996 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939569+0000", 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031548890000000001 2024-09-16T11:13:39.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939609+0000", 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0225000000000002e-05 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939650+0000", 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0500000000000002e-05 2024-09-16T11:13:39.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939700+0000", 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0161000000000003e-05 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14026 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915778+0000", 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.4140811260000001, 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025917228, 2024-09-16T11:13:39.437 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14026 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915778+0000", 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915778+0000", 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915779+0000", 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915785+0000", 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915798+0000", 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3123e-05 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915805+0000", 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2690000000000002e-06 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.937533+0000", 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021727847000000002 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.937657+0000", 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012432099999999999 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.938233+0000", 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057565700000000003 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939892+0000", 2024-09-16T11:13:39.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016591939999999999 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.941608+0000", 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0017153520000000001 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.941661+0000", 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3585000000000003e-05 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.941695+0000", 2024-09-16T11:13:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4088000000000003e-05 2024-09-16T11:13:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14060 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.960478+0000", 2024-09-16T11:13:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3693812479999998, 2024-09-16T11:13:39.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023397426999999998, 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14060 2024-09-16T11:13:39.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960478+0000", 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960478+0000", 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960479+0000", 2024-09-16T11:13:39.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9699999999999996e-07 2024-09-16T11:13:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960482+0000", 2024-09-16T11:13:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9299999999999999e-06 2024-09-16T11:13:39.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960494+0000", 2024-09-16T11:13:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2826999999999999e-05 2024-09-16T11:13:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960507+0000", 2024-09-16T11:13:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2875e-05 2024-09-16T11:13:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.979744+0000", 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019236559 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.979810+0000", 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6209000000000003e-05 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.980108+0000", 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029817999999999999 2024-09-16T11:13:39.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983011+0000", 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029025710000000001 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983742+0000", 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073078499999999996 2024-09-16T11:13:39.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983808+0000", 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6161000000000006e-05 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983875+0000", 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7732999999999997e-05 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14061 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.960512+0000", 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3693469489999996, 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025837055000000001, 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14061 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960512+0000", 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960512+0000", 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960513+0000", 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999995e-07 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960516+0000", 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999999e-06 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960524+0000", 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5000000000000002e-06 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960528+0000", 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.33e-06 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.981187+0000", 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020658826000000002 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.981286+0000", 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8914999999999997e-05 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.981569+0000", 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028276100000000003 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983719+0000", 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021504060000000001 2024-09-16T11:13:39.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986228+0000", 2024-09-16T11:13:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002508906 2024-09-16T11:13:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986298+0000", 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9985000000000002e-05 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986349+0000", 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1508999999999998e-05 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14062 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.960534+0000", 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.3693250590000003, 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025912858, 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14062 2024-09-16T11:13:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960534+0000", 2024-09-16T11:13:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960534+0000", 2024-09-16T11:13:39.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960535+0000", 2024-09-16T11:13:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:39.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960539+0000", 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.754e-06 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960560+0000", 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1228000000000001e-05 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960575+0000", 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5585000000000001e-05 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.982114+0000", 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021538976000000001 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.982184+0000", 2024-09-16T11:13:39.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9476999999999999e-05 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.982497+0000", 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031305200000000002 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.984011+0000", 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015143489999999999 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986363+0000", 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002351924 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986409+0000", 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6195000000000001e-05 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986447+0000", 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7775999999999999e-05 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.473 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14159 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.082343+0000", 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.2475164919999999, 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022780277000000002, 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:39.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14159 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082343+0000", 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082343+0000", 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:39.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082344+0000", 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082351+0000", 2024-09-16T11:13:39.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.703e-06 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082366+0000", 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5760000000000002e-05 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082377+0000", 2024-09-16T11:13:39.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0519999999999999e-05 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.101551+0000", 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01917367 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.101608+0000", 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7265000000000001e-05 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.101836+0000", 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000227798 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.104258+0000", 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024227659999999998 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.105028+0000", 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00076920000000000005 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.105085+0000", 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6993000000000001e-05 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.105123+0000", 2024-09-16T11:13:39.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8479000000000002e-05 2024-09-16T11:13:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:39.481 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:39.481 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:39.482 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:39.483 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone9 from 0xe8f8620 (overlap 0xc4a6be2) to 0x0 2024-09-16T11:13:39.483 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:39.477+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:39.490 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-16T11:13:39.583 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:39.577+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:39.590 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:39.590 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:39.590 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:39.590 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:39.600 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:39.685 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:39.677+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:39.694 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:39.694 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:39.695 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:39.695 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:39.695 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:39.703 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:39.785 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:39.780+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:39.810 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:39.810 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:39.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:891 4.9 4:92febedc:::rbd_data.11b585795d6b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 8388608~16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.875936518, 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015304566, 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.811 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 891 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929045+0000", 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.929047+0000", 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4729999999999999e-06 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933394+0000", 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043469529999999998 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933425+0000", 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1238999999999997e-05 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933441+0000", 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5719999999999999e-05 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933522+0000", 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1339000000000002e-05 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933648+0000", 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000125883 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.933908+0000", 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00026008699999999999 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.942070+0000", 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0081615249999999993 2024-09-16T11:13:39.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944253+0000", 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0021831910000000001 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944310+0000", 2024-09-16T11:13:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7241999999999998e-05 2024-09-16T11:13:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.944350+0000", 2024-09-16T11:13:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9913999999999998e-05 2024-09-16T11:13:39.818 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.818 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.818 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.818 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:4001 4.1b 4:dba71535:::rbd_data.11b5b93336d2.0000000000000016:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:39.819 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:39.819 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.053562246, 2024-09-16T11:13:39.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.027255873, 2024-09-16T11:13:39.819 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.819 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.819 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.820 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 4001 2024-09-16T11:13:39.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.820 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:39.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:39.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751422+0000", 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2469999999999998e-06 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751434+0000", 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1761e-05 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751469+0000", 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5577000000000002e-05 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751483+0000", 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3570999999999999e-05 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751628+0000", 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014536400000000001 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751857+0000", 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000228452 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.758485+0000", 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066286000000000001 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778412+0000", 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019927027 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778513+0000", 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010084 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778611+0000", 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8432999999999998e-05 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778675+0000", 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4000999999999999e-05 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15969 4.d 4:b69afd92:::rbd_data.11b58a761a59.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:39.828 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.72589156, 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044609673000000002, 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15969 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079092+0000", 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.17e-06 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079104+0000", 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079140+0000", 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5648000000000002e-05 2024-09-16T11:13:39.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079152+0000", 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2636999999999999e-05 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079270+0000", 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011812200000000001 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079499+0000", 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022818 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.082238+0000", 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027396949999999999 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123504+0000", 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041265556000000002 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123526+0000", 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1971999999999999e-05 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123623+0000", 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7347000000000002e-05 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123700+0000", 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6786000000000006e-05 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.834 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15970 4.d 4:b7b5bf19:::rbd_data.11b58a761a59.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7255868979999995, 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044492102999999998, 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15970 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079397+0000", 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.793e-06 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079405+0000", 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079421+0000", 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6201e-05 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079429+0000", 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.092788+0000", 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013358761 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.093013+0000", 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000225687 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.098959+0000", 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059459059999999999 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123735+0000", 2024-09-16T11:13:39.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024775755999999999 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123753+0000", 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8065999999999999e-05 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123836+0000", 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-05 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123887+0000", 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0559000000000002e-05 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:15969 4.d e47/36 ondisk, result = 0)", 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.7150702300000003, 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033797389999999997, 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089913+0000", 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.04e-06 2024-09-16T11:13:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089921+0000", 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2309999999999996e-06 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089941+0000", 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0494000000000001e-05 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089948+0000", 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4779999999999996e-06 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123470+0000", 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033522226000000002 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123488+0000", 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8269000000000001e-05 2024-09-16T11:13:39.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123709+0000", 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220652 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22473 4.13 4:cf41caf9:::rbd_data.11b52f8117af.000000000000000b:a [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.410880+0000", 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.394101341, 2024-09-16T11:13:39.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011888360000000001, 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22473 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410880+0000", 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410880+0000", 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410882+0000", 2024-09-16T11:13:39.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1519999999999999e-06 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410886+0000", 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5260000000000004e-06 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410899+0000", 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3198e-05 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410941+0000", 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1405000000000001e-05 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422580+0000", 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011638928 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422645+0000", 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5456999999999993e-05 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422769+0000", 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012369400000000001 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.850 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22478 4.1b 4:d9e48d5a:::rbd_data.11b52f8117af.000000000000000a:a [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.410981+0000", 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3940006839999999, 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011906758999999999, 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22478 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.851 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410981+0000", 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410981+0000", 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410982+0000", 2024-09-16T11:13:39.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3039999999999999e-06 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410986+0000", 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.89e-06 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.410998+0000", 2024-09-16T11:13:39.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1936000000000001e-05 2024-09-16T11:13:39.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411006+0000", 2024-09-16T11:13:39.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.447e-06 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422602+0000", 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011596715 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422704+0000", 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101463 2024-09-16T11:13:39.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422888+0000", 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018400400000000001 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22479 4.13 4:cf41caf9:::rbd_data.11b52f8117af.000000000000000b:a [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411030+0000", 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.393951612, 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012318188000000001, 2024-09-16T11:13:39.856 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22479 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411030+0000", 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411030+0000", 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411031+0000", 2024-09-16T11:13:39.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6799999999999999e-07 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411036+0000", 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4950000000000002e-06 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411049+0000", 2024-09-16T11:13:39.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3668e-05 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411058+0000", 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6570000000000003e-06 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422806+0000", 2024-09-16T11:13:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011748084000000001 2024-09-16T11:13:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422891+0000", 2024-09-16T11:13:39.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5195e-05 2024-09-16T11:13:39.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423348+0000", 2024-09-16T11:13:39.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045722100000000001 2024-09-16T11:13:39.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.863 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.864 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22504 4.1b 4:d9e48d5a:::rbd_data.11b52f8117af.000000000000000a:a [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.864 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411134+0000", 2024-09-16T11:13:39.864 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3938481730000001, 2024-09-16T11:13:39.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012064159, 2024-09-16T11:13:39.864 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.864 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.865 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.865 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.865 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.865 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22504 2024-09-16T11:13:39.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.865 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411134+0000", 2024-09-16T11:13:39.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411134+0000", 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411134+0000", 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8700000000000005e-07 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411138+0000", 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9360000000000001e-06 2024-09-16T11:13:39.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411175+0000", 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6856000000000001e-05 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411181+0000", 2024-09-16T11:13:39.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:13:39.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.422926+0000", 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011745597999999999 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423089+0000", 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162244 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423198+0000", 2024-09-16T11:13:39.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109212 2024-09-16T11:13:39.872 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.872 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.872 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22505 4.13 4:cf41caf9:::rbd_data.11b52f8117af.000000000000000b:a [read 212992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411187+0000", 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3937946189999999, 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012428323999999999, 2024-09-16T11:13:39.873 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22505 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411187+0000", 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411187+0000", 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411188+0000", 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759999999999999e-06 2024-09-16T11:13:39.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411191+0000", 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1080000000000001e-06 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411197+0000", 2024-09-16T11:13:39.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2290000000000002e-06 2024-09-16T11:13:39.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411203+0000", 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0580000000000003e-06 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423427+0000", 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01222431 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423514+0000", 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6953000000000006e-05 2024-09-16T11:13:39.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423616+0000", 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010158999999999999 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22510 4.1b 4:d9e48d5a:::rbd_data.11b52f8117af.000000000000000a:a [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411208+0000", 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3937739730000001, 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012404414000000001, 2024-09-16T11:13:39.882 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22510 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411208+0000", 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411208+0000", 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411208+0000", 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4799999999999999e-07 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411211+0000", 2024-09-16T11:13:39.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4279999999999999e-06 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411217+0000", 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8649999999999998e-06 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411223+0000", 2024-09-16T11:13:39.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2559999999999999e-06 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423404+0000", 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012181496999999999 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423499+0000", 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4951999999999995e-05 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423612+0000", 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011296800000000001 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22511 4.13 4:cf41caf9:::rbd_data.11b52f8117af.000000000000000b:a [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411226+0000", 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.393756153, 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012601167999999999, 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.888 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22511 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411226+0000", 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411226+0000", 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411226+0000", 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411228+0000", 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.79e-06 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411237+0000", 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8389999999999996e-06 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411241+0000", 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.831e-06 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423649+0000", 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012408033000000001 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423724+0000", 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.436e-05 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423827+0000", 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010301700000000001 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.892 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22528 4.1b 4:d9e48d5a:::rbd_data.11b52f8117af.000000000000000a:a [read 188416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411263+0000", 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3937191389999999, 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01258039, 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.893 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22528 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411263+0000", 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411263+0000", 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411263+0000", 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411266+0000", 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4360000000000001e-06 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411271+0000", 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8440000000000003e-06 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411274+0000", 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423677+0000", 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012402231999999999 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423754+0000", 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7303000000000002e-05 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423843+0000", 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9254999999999998e-05 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22529 4.13 4:cf41caf9:::rbd_data.11b52f8117af.000000000000000b:a [read 188416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411277+0000", 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3937049049999999, 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012736954, 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22529 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411277+0000", 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411277+0000", 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411278+0000", 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9200000000000001e-07 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411286+0000", 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3040000000000005e-06 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411293+0000", 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3889999999999999e-06 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411298+0000", 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6399999999999996e-06 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423865+0000", 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012566805 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423925+0000", 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0690999999999999e-05 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.424014+0000", 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8533000000000001e-05 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22542 4.1b 4:d9e48d5a:::rbd_data.11b52f8117af.000000000000000a:a [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411319+0000", 2024-09-16T11:13:39.902 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.3936625030000001, 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013290208, 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22542 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411319+0000", 2024-09-16T11:13:39.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411319+0000", 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411320+0000", 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411322+0000", 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0030000000000001e-06 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411330+0000", 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6799999999999993e-06 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411334+0000", 2024-09-16T11:13:39.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6500000000000004e-06 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423884+0000", 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012549538000000001 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.423957+0000", 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3640999999999995e-05 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.424610+0000", 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00065218199999999996 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22543 4.13 4:cf41caf9:::rbd_data.11b52f8117af.000000000000000b:a [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.411338+0000", 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.393643623, 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012904591999999999, 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.907 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22543 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411338+0000", 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411338+0000", 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411339+0000", 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.96e-07 2024-09-16T11:13:39.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411340+0000", 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.824e-06 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411345+0000", 2024-09-16T11:13:39.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2209999999999999e-06 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.411349+0000", 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.424040+0000", 2024-09-16T11:13:39.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01269143 2024-09-16T11:13:39.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.424116+0000", 2024-09-16T11:13:39.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5631999999999997e-05 2024-09-16T11:13:39.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.424243+0000", 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012679600000000001 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26603 4.1a 4:5aec3350:::rbd_data.11b5499bda95.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.847753+0000", 2024-09-16T11:13:39.914 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.95722883400000003, 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019919959000000001, 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26603 2024-09-16T11:13:39.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.847753+0000", 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.847753+0000", 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.847755+0000", 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9700000000000002e-06 2024-09-16T11:13:39.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.851741+0000", 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039856960000000004 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.851773+0000", 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2190000000000002e-05 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.851785+0000", 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1773000000000001e-05 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.851870+0000", 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5363000000000002e-05 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.852098+0000", 2024-09-16T11:13:39.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000227963 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.853077+0000", 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00097938600000000006 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.862044+0000", 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089661910000000001 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.867590+0000", 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0055467290000000002 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.867636+0000", 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5473999999999999e-05 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.867673+0000", 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7224000000000001e-05 2024-09-16T11:13:39.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26626 4.18 4:1e0f541f:::rbd_data.11b5499bda95.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 29639583~50972] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.877149+0000", 2024-09-16T11:13:39.924 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.927832977, 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012911908999999999, 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26626 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.877149+0000", 2024-09-16T11:13:39.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.877149+0000", 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.877151+0000", 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3520000000000001e-06 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.878989+0000", 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0018378419999999999 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.879023+0000", 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4329000000000003e-05 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.879036+0000", 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3016999999999999e-05 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.879096+0000", 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9497999999999999e-05 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.879273+0000", 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017735099999999999 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.879815+0000", 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054128100000000001 2024-09-16T11:13:39.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.884907+0000", 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0050923219999999998 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.889921+0000", 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0050138370000000002 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.889997+0000", 2024-09-16T11:13:39.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6067000000000002e-05 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.890061+0000", 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4012999999999999e-05 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26656 4.9 4:95b731cd:::rbd_data.11b5499bda95.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.85408189199999995, 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.054121295999999999, 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26656 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950902+0000", 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950914+0000", 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1724e-05 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950943+0000", 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9621e-05 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950956+0000", 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.238e-05 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951090+0000", 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013472399999999999 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951475+0000", 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000384378 2024-09-16T11:13:39.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.964699+0000", 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013224217999999999 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004806+0000", 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.040106941 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004820+0000", 2024-09-16T11:13:39.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3909e-05 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004895+0000", 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4905000000000002e-05 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.005021+0000", 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012639300000000001 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.936 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:26656 4.9 e50/31 ondisk, result = 0)", 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.990830+0000", 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81415153200000001, 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013979081000000001, 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.990830+0000", 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.990830+0000", 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.990831+0000", 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1039999999999999e-06 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.990838+0000", 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.990859+0000", 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0934e-05 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.990868+0000", 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0929999999999992e-06 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004786+0000", 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013918546 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004797+0000", 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0387999999999999e-05 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004809+0000", 2024-09-16T11:13:39.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2581e-05 2024-09-16T11:13:39.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.941 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:39.941 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:39.941 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:39.941 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:39.943 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:39.943 INFO:teuthology.orchestra.run.smithi162.stdout:1354 punch from 0x7903993 to 0x7904ba9, (0x1216 bytes) 2024-09-16T11:13:39.943 INFO:teuthology.orchestra.run.smithi162.stdout:1355 write 0x3b4fe65 thru 0x3b53b71 (0x3d0d bytes) 2024-09-16T11:13:39.943 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:39.936+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:39.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:39.947 INFO:teuthology.orchestra.run.smithi162.stdout:1356 read 0x3a435ae thru 0x3a48746 (0x5199 bytes) 2024-09-16T11:13:39.949 INFO:teuthology.orchestra.run.smithi162.stdout:1358 read 0x3a623f0 thru 0x3a6d57f (0xb190 bytes) 2024-09-16T11:13:39.952 INFO:teuthology.orchestra.run.smithi162.stdout:1360 punch from 0x94e8eab to 0x94f2b91, (0x9ce6 bytes) 2024-09-16T11:13:39.952 INFO:teuthology.orchestra.run.smithi162.stdout:1361 write 0xc5d17fa thru 0xc5e168c (0xfe93 bytes) 2024-09-16T11:13:39.962 INFO:teuthology.orchestra.run.smithi162.stdout:1362 read 0x67fbb04 thru 0x67fefb1 (0x34ae bytes) 2024-09-16T11:13:39.962 INFO:teuthology.orchestra.run.smithi162.stdout:1365 punch from 0x8ab9e57 to 0x8ac39be, (0x9b67 bytes) 2024-09-16T11:13:39.963 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:39.963 INFO:teuthology.orchestra.run.smithi162.stdout:1368 read 0x621bd6e thru 0x621ffe0 (0x4273 bytes) 2024-09-16T11:13:39.963 INFO:teuthology.orchestra.run.smithi162.stdout:1369 write 0xa40c470 thru 0xa41134c (0x4edd bytes) 2024-09-16T11:13:39.970 INFO:teuthology.orchestra.run.smithi162.stdout:1370 read 0x6b042ab thru 0x6b1290b (0xe661 bytes) 2024-09-16T11:13:39.970 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:39.970 INFO:teuthology.orchestra.run.smithi162.stdout:1372 read 0x4c17e3c thru 0x4c1ee8a (0x704f bytes) 2024-09-16T11:13:39.971 INFO:teuthology.orchestra.run.smithi162.stdout:1374 write 0xbbcde60 thru 0xbbd1b89 (0x3d2a bytes) 2024-09-16T11:13:39.980 INFO:teuthology.orchestra.run.smithi162.stdout:1375 write 0xeb6350 thru 0xebc400 (0x60b1 bytes) 2024-09-16T11:13:39.989 INFO:teuthology.orchestra.run.smithi162.stdout:1376 writesame 0x7bba61a thru 0x7bbb045 data_size 0x1f(0xa2c bytes) 2024-09-16T11:13:39.991 INFO:teuthology.orchestra.run.smithi162.stdout:1378 read 0xce592b9 thru 0xce691a3 (0xfeeb bytes) 2024-09-16T11:13:40.016 INFO:teuthology.orchestra.run.smithi162.stdout:1379 trunc from 0xe8f8620 to 0xa6d0f5 2024-09-16T11:13:40.044 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:40.036+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:40.048 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:40.048 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:40.049 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:40.050 INFO:teuthology.orchestra.run.smithi162.stdout:1383 read 0x2877ca thru 0x29207e (0xa8b5 bytes) 2024-09-16T11:13:40.050 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.050 INFO:teuthology.orchestra.run.smithi162.stdout:1385 punch from 0x8a4cef to 0x8a7219, (0x252a bytes) 2024-09-16T11:13:40.050 INFO:teuthology.orchestra.run.smithi162.stdout:1386 read 0x287deb thru 0x29700d (0xf223 bytes) 2024-09-16T11:13:40.051 INFO:teuthology.orchestra.run.smithi162.stdout:1387 read 0x7ddc0 thru 0x8bfff (0xe240 bytes) 2024-09-16T11:13:40.051 INFO:teuthology.orchestra.run.smithi162.stdout:1388 punch from 0x3c820e to 0x3cfa28, (0x781a bytes) 2024-09-16T11:13:40.051 INFO:teuthology.orchestra.run.smithi162.stdout:1389 trunc from 0xa6d0f5 to 0xc49c88 2024-09-16T11:13:40.058 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:40.086 INFO:teuthology.orchestra.run.smithi162.stdout:1391 write 0xd210c5b thru 0xd212f61 (0x2307 bytes) 2024-09-16T11:13:40.096 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:40.108 INFO:teuthology.orchestra.run.smithi162.stdout:1392 trunc from 0xd212f62 to 0x2a7edf2 2024-09-16T11:13:40.119 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:40.130 INFO:teuthology.orchestra.run.smithi162.stdout:1393 trunc from 0x2a7edf2 to 0xa02f06 2024-09-16T11:13:40.139 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:40.145 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:40.140+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:40.153 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:40.153 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:40.153 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:40.153 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:40.153 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:40.157 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:40.160 INFO:teuthology.orchestra.run.smithi162.stdout:1394 trunc from 0xa02f06 to 0xcc3fe1a 2024-09-16T11:13:40.163 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:40.167 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.167 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.167 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.167 INFO:teuthology.orchestra.run.smithi162.stdout:1398 writesame 0x2ddfaa thru 0x2e6543 data_size 0x31(0x859a bytes) 2024-09-16T11:13:40.170 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.179 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:40.188 INFO:teuthology.orchestra.run.smithi162.stdout:1401 trunc from 0xcc3fe1a to 0x7fa8b 2024-09-16T11:13:40.202 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:40.226 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:40.245 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:40.245 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:40.237+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:40.247 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:40.248 INFO:teuthology.orchestra.run.smithi162.stdout:1402 write 0xa4e8567 thru 0xa4f55f0 (0xd08a bytes) 2024-09-16T11:13:40.254 INFO:teuthology.orchestra.run.smithi162.stdout:1405 read 0x5add26d thru 0x5ae1f88 (0x4d1c bytes) 2024-09-16T11:13:40.256 INFO:teuthology.orchestra.run.smithi162.stdout:1406 trunc from 0xa4f55f1 to 0x974a5f1 2024-09-16T11:13:40.269 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.269 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.269 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.270 INFO:teuthology.orchestra.run.smithi162.stdout:1408 punch from 0x1e38d8 to 0x1e8227, (0x494f bytes) 2024-09-16T11:13:40.270 INFO:teuthology.orchestra.run.smithi162.stdout:1409 write 0xe18d74 thru 0xe1c2f1 (0x357e bytes) 2024-09-16T11:13:40.274 INFO:teuthology.orchestra.run.smithi162.stdout:1410 write 0x2820d4e thru 0x28221ab (0x145e bytes) 2024-09-16T11:13:40.280 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.281 INFO:teuthology.orchestra.run.smithi162.stdout:1412 read 0x21807ae thru 0x2188ad3 (0x8326 bytes) 2024-09-16T11:13:40.281 INFO:teuthology.orchestra.run.smithi162.stdout:1414 punch from 0x9049113 to 0x9052527, (0x9414 bytes) 2024-09-16T11:13:40.281 INFO:teuthology.orchestra.run.smithi162.stdout:1415 write 0x69c7af2 thru 0x69cda18 (0x5f27 bytes) 2024-09-16T11:13:40.283 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:40.292 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:13:40.292 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.309 INFO:teuthology.orchestra.run.smithi162.stdout:1419 write 0xd4aadbd thru 0xd4b7150 (0xc394 bytes) 2024-09-16T11:13:40.323 INFO:teuthology.orchestra.run.smithi162.stdout:1420 trunc from 0xd4b7151 to 0x8e96b7d 2024-09-16T11:13:40.342 INFO:teuthology.orchestra.run.smithi162.stdout:1421 punch from 0x879ca7c to 0x87aa3d5, (0xd959 bytes) 2024-09-16T11:13:40.343 INFO:teuthology.orchestra.run.smithi162.stdout:1423 read 0x29848b thru 0x29cca0 (0x4816 bytes) 2024-09-16T11:13:40.343 INFO:teuthology.orchestra.run.smithi162.stdout:1425 writesame 0x63948e5 thru 0x639e1db data_size 0x3(0x98f7 bytes) 2024-09-16T11:13:40.346 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:40.340+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:40.360 INFO:teuthology.orchestra.run.smithi162.stdout:1426 read 0x76885d4 thru 0x769502e (0xca5b bytes) 2024-09-16T11:13:40.370 INFO:teuthology.orchestra.run.smithi162.stdout:1428 writesame 0x9c830dc thru 0x9c91c3a data_size 0x75(0xeb5f bytes) 2024-09-16T11:13:40.383 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.383 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.386 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:40.386 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:40.386 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T11:13:40.387 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:13:40.387 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:13:40.387 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T11:13:40.388 INFO:teuthology.orchestra.run.smithi162.stdout:1429 trunc from 0x9c91c3b to 0x1ebe304 2024-09-16T11:13:40.392 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:40.431 INFO:teuthology.orchestra.run.smithi162.stdout:1431 write 0x88f6ad2 thru 0x88ff6d5 (0x8c04 bytes) 2024-09-16T11:13:40.436 INFO:teuthology.orchestra.run.smithi162.stdout:1432 writesame 0x44926e3 thru 0x449d621 data_size 0xd(0xaf3f bytes) 2024-09-16T11:13:40.447 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:40.440+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:40.454 INFO:teuthology.orchestra.run.smithi162.stdout:1433 write 0x9b7ac86 thru 0x9b8203b (0x73b6 bytes) 2024-09-16T11:13:40.462 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.463 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.463 INFO:teuthology.orchestra.run.smithi162.stdout:1439 write 0x584dd45 thru 0x5854f43 (0x71ff bytes) 2024-09-16T11:13:40.471 INFO:teuthology.orchestra.run.smithi162.stdout:1440 write 0x1dcf60d thru 0x1dd318c (0x3b80 bytes) 2024-09-16T11:13:40.474 INFO:teuthology.orchestra.run.smithi162.stdout:1442 writesame 0x5064b0f thru 0x506f188 data_size 0x3(0xa67a bytes) 2024-09-16T11:13:40.489 INFO:teuthology.orchestra.run.smithi162.stdout:1443 read 0x1b87aa3 thru 0x1b8de06 (0x6364 bytes) 2024-09-16T11:13:40.490 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:40.490 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:13:40.490 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:40.491 INFO:teuthology.orchestra.run.smithi162.stdout:1444 writesame 0x93c6a06 thru 0x93d502c data_size 0x13(0xe627 bytes) 2024-09-16T11:13:40.492 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.492 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.493 INFO:teuthology.orchestra.run.smithi162.stdout:1445 punch from 0x1c8fb72 to 0x1c9d9f9, (0xde87 bytes) 2024-09-16T11:13:40.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:40.504 INFO:teuthology.orchestra.run.smithi162.stdout:1446 write 0xb9f9677 thru 0xba02235 (0x8bbf bytes) 2024-09-16T11:13:40.526 INFO:teuthology.orchestra.run.smithi162.stdout:1448 write 0xee421e1 thru 0xee51f6d (0xfd8d bytes) 2024-09-16T11:13:40.531 INFO:teuthology.orchestra.run.smithi162.stdout:1450 writesame 0x2551fc1 thru 0x255b60b data_size 0x1b(0x964b bytes) 2024-09-16T11:13:40.538 INFO:teuthology.orchestra.run.smithi162.stdout:1452 write 0xaa7729 thru 0xaabef8 (0x47d0 bytes) 2024-09-16T11:13:40.544 INFO:teuthology.orchestra.run.smithi162.stdout:1453 write 0xe9eb58 thru 0xea634f (0x77f8 bytes) 2024-09-16T11:13:40.548 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:40.540+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:40.549 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.558 INFO:teuthology.orchestra.run.smithi162.stdout:1458 trunc from 0xee51f6e to 0xa7f5e21 2024-09-16T11:13:40.579 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.595 INFO:teuthology.orchestra.run.smithi162.stdout:1460 writesame 0xe206370 thru 0xe20b33a data_size 0xb(0x4fcb bytes) 2024-09-16T11:13:40.595 INFO:teuthology.orchestra.run.smithi162.stdout:1461 read 0xc88e84c thru 0xc89623d (0x79f2 bytes) 2024-09-16T11:13:40.595 INFO:teuthology.orchestra.run.smithi162.stdout:1462 writesame 0x77c1706 thru 0x77c3d4f data_size 0x1d(0x264a bytes) 2024-09-16T11:13:40.599 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.599 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.601 INFO:teuthology.orchestra.run.smithi162.stdout:1463 writesame 0x3d07778 thru 0x3d0da68 data_size 0x1(0x62f1 bytes) 2024-09-16T11:13:40.608 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:40.625 INFO:teuthology.orchestra.run.smithi162.stdout:1465 read 0xac5dc9a thru 0xac69124 (0xb48b bytes) 2024-09-16T11:13:40.626 INFO:teuthology.orchestra.run.smithi162.stdout:1467 read 0x814d17e thru 0x81534db (0x635e bytes) 2024-09-16T11:13:40.627 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.627 INFO:teuthology.orchestra.run.smithi162.stdout:1470 read 0xbe78109 thru 0xbe84fd9 (0xced1 bytes) 2024-09-16T11:13:40.627 INFO:teuthology.orchestra.run.smithi162.stdout:1471 punch from 0x8241dd to 0x82a542, (0x6365 bytes) 2024-09-16T11:13:40.627 INFO:teuthology.orchestra.run.smithi162.stdout:1474 read 0x6b17fc8 thru 0x6b1925a (0x1293 bytes) 2024-09-16T11:13:40.627 INFO:teuthology.orchestra.run.smithi162.stdout:1475 writesame 0x44b764 thru 0x457b1f data_size 0x2(0xc3bc bytes) 2024-09-16T11:13:40.647 INFO:teuthology.orchestra.run.smithi162.stdout:1477 read 0x53c595 thru 0x54bf83 (0xf9ef bytes) 2024-09-16T11:13:40.648 INFO:teuthology.orchestra.run.smithi162.stdout:1478 write 0xdbda202 thru 0xdbded90 (0x4b8f bytes) 2024-09-16T11:13:40.649 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:40.644+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:40.654 INFO:teuthology.orchestra.run.smithi162.stdout:1479 write 0x28a8599 thru 0x28b5e47 (0xd8af bytes) 2024-09-16T11:13:40.657 INFO:teuthology.orchestra.run.smithi162.stdout:1480 writesame 0x761fa49 thru 0x76271cf data_size 0x1(0x7787 bytes) 2024-09-16T11:13:40.686 INFO:teuthology.orchestra.run.smithi162.stdout:1485 write 0x128f900 thru 0x129d631 (0xdd32 bytes) 2024-09-16T11:13:40.689 INFO:teuthology.orchestra.run.smithi162.stdout:1486 write 0x83ac7d6 thru 0x83ae033 (0x185e bytes) 2024-09-16T11:13:40.694 INFO:teuthology.orchestra.run.smithi162.stdout:1488 write 0xf0925e thru 0xf0d8c5 (0x4668 bytes) 2024-09-16T11:13:40.697 INFO:teuthology.orchestra.run.smithi162.stdout:1489 punch from 0x3105c5a to 0x3112d18, (0xd0be bytes) 2024-09-16T11:13:40.697 INFO:teuthology.orchestra.run.smithi162.stdout:1490 read 0x2752d81 thru 0x275f9bd (0xcc3d bytes) 2024-09-16T11:13:40.698 INFO:teuthology.orchestra.run.smithi162.stdout:1491 read 0xaa2bdd3 thru 0xaa3a3dc (0xe60a bytes) 2024-09-16T11:13:40.699 INFO:teuthology.orchestra.run.smithi162.stdout:1492 punch from 0x4191eb1 to 0x419fa92, (0xdbe1 bytes) 2024-09-16T11:13:40.699 INFO:teuthology.orchestra.run.smithi162.stdout:1494 punch from 0x71b6d39 to 0x71bf312, (0x85d9 bytes) 2024-09-16T11:13:40.699 INFO:teuthology.orchestra.run.smithi162.stdout:1497 punch from 0x130e9a9 to 0x131529c, (0x68f3 bytes) 2024-09-16T11:13:40.699 INFO:teuthology.orchestra.run.smithi162.stdout:1498 write 0x5307a49 thru 0x530ab76 (0x312e bytes) 2024-09-16T11:13:40.702 INFO:teuthology.orchestra.run.smithi162.stdout:1499 punch from 0x7ac3fb0 to 0x7ac7b34, (0x3b84 bytes) 2024-09-16T11:13:40.702 INFO:teuthology.orchestra.run.smithi162.stdout:1500 write 0x3bdbaa3 thru 0x3bdc572 (0xad0 bytes) 2024-09-16T11:13:40.708 INFO:teuthology.orchestra.run.smithi162.stdout:1501 write 0x67b7cfc thru 0x67c1d3a (0xa03f bytes) 2024-09-16T11:13:40.712 INFO:teuthology.orchestra.run.smithi162.stdout:1502 writesame 0xe0002c5 thru 0xe00ac24 data_size 0x50(0xa960 bytes) 2024-09-16T11:13:40.712 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.713 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.717 INFO:teuthology.orchestra.run.smithi162.stdout:1503 write 0x9e2e937 thru 0x9e332b5 (0x497f bytes) 2024-09-16T11:13:40.720 INFO:teuthology.orchestra.run.smithi162.stdout:1504 write 0x13f1212 thru 0x13f18dc (0x6cb bytes) 2024-09-16T11:13:40.722 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:40.723 INFO:teuthology.orchestra.run.smithi162.stdout:1505 read 0x97b18d3 thru 0x97c0639 (0xed67 bytes) 2024-09-16T11:13:40.723 INFO:teuthology.orchestra.run.smithi162.stdout:1508 read 0x8a20344 thru 0x8a2e5a8 (0xe265 bytes) 2024-09-16T11:13:40.723 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.723 INFO:teuthology.orchestra.run.smithi162.stdout:1510 write 0xa0934d3 thru 0xa09e356 (0xae84 bytes) 2024-09-16T11:13:40.726 INFO:teuthology.orchestra.run.smithi162.stdout:1511 write 0x911acef thru 0x911deaa (0x31bc bytes) 2024-09-16T11:13:40.731 INFO:teuthology.orchestra.run.smithi162.stdout:1512 write 0x6da0c26 thru 0x6dab256 (0xa631 bytes) 2024-09-16T11:13:40.734 INFO:teuthology.orchestra.run.smithi162.stdout:1514 read 0x9b1feb1 thru 0x9b21304 (0x1454 bytes) 2024-09-16T11:13:40.735 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.735 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.735 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.735 INFO:teuthology.orchestra.run.smithi162.stdout:1518 writesame 0x524e39e thru 0x525d2bf data_size 0x13(0xef22 bytes) 2024-09-16T11:13:40.741 INFO:teuthology.orchestra.run.smithi162.stdout:1520 punch from 0xc2c0bb7 to 0xc2c358e, (0x29d7 bytes) 2024-09-16T11:13:40.741 INFO:teuthology.orchestra.run.smithi162.stdout:1521 write 0xa2f0290 thru 0xa2f5141 (0x4eb2 bytes) 2024-09-16T11:13:40.747 INFO:teuthology.orchestra.run.smithi162.stdout:1522 flatten 2024-09-16T11:13:40.750 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:40.744+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:40.759 INFO:teuthology.orchestra.run.smithi162.stdout:1524 writesame 0x61607c8 thru 0x61670c3 data_size 0x2(0x68fc bytes) 2024-09-16T11:13:40.785 INFO:teuthology.orchestra.run.smithi162.stdout:1525 trunc from 0xe20b33b to 0xa7954c3 2024-09-16T11:13:40.809 INFO:teuthology.orchestra.run.smithi162.stdout:1528 writesame 0x8debaf0 thru 0x8deed6d data_size 0x17(0x327e bytes) 2024-09-16T11:13:40.824 INFO:teuthology.orchestra.run.smithi162.stdout:1529 write 0xba37dea thru 0xba4114e (0x9365 bytes) 2024-09-16T11:13:40.825 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.825 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.832 INFO:teuthology.orchestra.run.smithi162.stdout:1530 trunc from 0xba4114f to 0xc669df9 2024-09-16T11:13:40.834 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:40.850 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:40.845+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:40.859 INFO:teuthology.orchestra.run.smithi162.stdout:1531 trunc from 0xc669df9 to 0x522d3ec 2024-09-16T11:13:40.882 INFO:teuthology.orchestra.run.smithi162.stdout:1532 write 0x2beb9c5 thru 0x2bece1e (0x145a bytes) 2024-09-16T11:13:40.885 INFO:teuthology.orchestra.run.smithi162.stdout:1533 read 0x58f3c6 thru 0x592759 (0x3394 bytes) 2024-09-16T11:13:40.887 INFO:teuthology.orchestra.run.smithi162.stdout:1534 trunc from 0x522d3ec to 0x3d11e91 2024-09-16T11:13:40.900 INFO:teuthology.orchestra.run.smithi162.stdout:1535 write 0x3001a73 thru 0x3007967 (0x5ef5 bytes) 2024-09-16T11:13:40.903 INFO:teuthology.orchestra.run.smithi162.stdout:1536 punch from 0xe3c338 to 0xe41c23, (0x58eb bytes) 2024-09-16T11:13:40.903 INFO:teuthology.orchestra.run.smithi162.stdout:1537 read 0x13ff926 thru 0x1408847 (0x8f22 bytes) 2024-09-16T11:13:40.932 INFO:teuthology.orchestra.run.smithi162.stdout:1538 write 0xee12974 thru 0xee176b8 (0x4d45 bytes) 2024-09-16T11:13:40.935 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:40.936 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:40.938 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:40.938 INFO:teuthology.orchestra.run.smithi162.stdout:1540 write 0x5dbb18d thru 0x5dbeac0 (0x3934 bytes) 2024-09-16T11:13:40.944 INFO:teuthology.orchestra.run.smithi162.stdout:1542 write 0xa6aeeed thru 0xa6b0014 (0x1128 bytes) 2024-09-16T11:13:40.945 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:40.950 INFO:teuthology.orchestra.run.smithi162.stdout:1544 read 0x442a0d7 thru 0x442f47c (0x53a6 bytes) 2024-09-16T11:13:40.950 INFO:teuthology.orchestra.run.smithi162.stdout:1547 write 0x11e21ca thru 0x11e925b (0x7092 bytes) 2024-09-16T11:13:40.951 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:40.944+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:40.956 INFO:teuthology.orchestra.run.smithi162.stdout:1548 clone 11 order 20 su 4096 sc 4 2024-09-16T11:13:41.052 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:41.049+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:41.052 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.052 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.061 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:41.153 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:41.149+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:41.213 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.213 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:41.254 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:41.248+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:41.332 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.332 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.342 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:41.354 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:41.349+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:41.443 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.443 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:41.455 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:41.448+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:41.556 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:41.553+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:41.556 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.556 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.565 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:41.574 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-16T11:13:41.590 INFO:teuthology.orchestra.run.smithi162.stdout:1549 punch from 0xaa6d63b to 0xaa72fef, (0x59b4 bytes) 2024-09-16T11:13:41.596 INFO:teuthology.orchestra.run.smithi162.stdout:1550 write 0xf3482b thru 0xf35216 (0x9ec bytes) 2024-09-16T11:13:41.650 INFO:teuthology.orchestra.run.smithi162.stdout:1551 write 0x4330bb0 thru 0x43323a7 (0x17f8 bytes) 2024-09-16T11:13:41.656 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:41.649+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:41.686 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.686 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.690 INFO:teuthology.orchestra.run.smithi162.stdout:1552 writesame 0x44fbf10 thru 0x4504175 data_size 0x1(0x8266 bytes) 2024-09-16T11:13:41.700 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:41.754 INFO:teuthology.orchestra.run.smithi162.stdout:1553 punch from 0xd714158 to 0xd7175c4, (0x346c bytes) 2024-09-16T11:13:41.754 INFO:teuthology.orchestra.run.smithi162.stdout:1554 read 0x61f5081 thru 0x61fd17e (0x80fe bytes) 2024-09-16T11:13:41.756 INFO:teuthology.orchestra.run.smithi162.stdout:1555 read 0x8a35197 thru 0x8a3f4dd (0xa347 bytes) 2024-09-16T11:13:41.756 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:41.749+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:41.759 INFO:teuthology.orchestra.run.smithi162.stdout:1556 punch from 0xa3bc148 to 0xa3c10ce, (0x4f86 bytes) 2024-09-16T11:13:41.759 INFO:teuthology.orchestra.run.smithi162.stdout:1559 writesame 0x1d274ea thru 0x1d2f733 data_size 0x6d(0x824a bytes) 2024-09-16T11:13:41.857 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:41.852+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:41.859 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:41.859 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:41.864 INFO:teuthology.orchestra.run.smithi162.stdout:1560 punch from 0x2020bdd to 0x2025466, (0x4889 bytes) 2024-09-16T11:13:41.864 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:41.864 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:41.864 INFO:teuthology.orchestra.run.smithi162.stdout:1564 read 0xc1fc03d thru 0xc1fed8f (0x2d53 bytes) 2024-09-16T11:13:41.866 INFO:teuthology.orchestra.run.smithi162.stdout:1566 write 0x4483281 thru 0x4492d53 (0xfad3 bytes) 2024-09-16T11:13:41.871 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:41.873 INFO:teuthology.orchestra.run.smithi162.stdout:1568 write 0xdfcf4ae thru 0xdfd74a4 (0x7ff7 bytes) 2024-09-16T11:13:41.907 INFO:teuthology.orchestra.run.smithi162.stdout:1569 punch from 0x5892b3f to 0x589f102, (0xc5c3 bytes) 2024-09-16T11:13:41.907 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:41.908 INFO:teuthology.orchestra.run.smithi162.stdout:1572 read 0xb930291 thru 0xb938a40 (0x87b0 bytes) 2024-09-16T11:13:41.908 INFO:teuthology.orchestra.run.smithi162.stdout:1573 read 0xe452921 thru 0xe45984a (0x6f2a bytes) 2024-09-16T11:13:41.912 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:41.930 INFO:teuthology.orchestra.run.smithi162.stdout:1575 trunc from 0xee176b9 to 0x36b9cdc 2024-09-16T11:13:41.964 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:41.952+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:42.008 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:42.008 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:42.017 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:42.019 INFO:teuthology.orchestra.run.smithi162.stdout:1576 read 0xb4e63b thru 0xb54db8 (0x677e bytes) 2024-09-16T11:13:42.030 INFO:teuthology.orchestra.run.smithi162.stdout:1578 read 0xa84b7b thru 0xa93d2e (0xf1b4 bytes) 2024-09-16T11:13:42.058 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:42.052+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:42.068 INFO:teuthology.orchestra.run.smithi162.stdout:1579 punch from 0x1ba1b29 to 0x1ba1bb9, (0x90 bytes) 2024-09-16T11:13:42.068 INFO:teuthology.orchestra.run.smithi162.stdout:1580 read 0x250ecef thru 0x251e048 (0xf35a bytes) 2024-09-16T11:13:42.083 INFO:teuthology.orchestra.run.smithi162.stdout:1581 trunc from 0x36b9cdc to 0x3373f51 2024-09-16T11:13:42.160 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:42.152+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:42.170 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:42.171 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:42.184 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:42.258 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:42.260 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:42.252+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:42.271 INFO:teuthology.orchestra.run.smithi162.stdout:1582 write 0x3aee571 thru 0x3aef65c (0x10ec bytes) 2024-09-16T11:13:42.279 INFO:teuthology.orchestra.run.smithi162.stdout:1584 punch from 0x47b78f to 0x47e026, (0x2897 bytes) 2024-09-16T11:13:42.304 INFO:teuthology.orchestra.run.smithi162.stdout:1585 writesame 0xe939781 thru 0xe9417ea data_size 0x3(0x806a bytes) 2024-09-16T11:13:42.304 INFO:teuthology.orchestra.run.smithi162.stdout:1586 read 0x76a165d thru 0x76b03d7 (0xed7b bytes) 2024-09-16T11:13:42.305 INFO:teuthology.orchestra.run.smithi162.stdout:1587 write 0x55cd1f6 thru 0x55d9bd7 (0xc9e2 bytes) 2024-09-16T11:13:42.306 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:13:42.318 INFO:teuthology.orchestra.run.smithi162.stdout:1589 writesame 0x686827e thru 0x6872bc6 data_size 0x29(0xa949 bytes) 2024-09-16T11:13:42.330 INFO:teuthology.orchestra.run.smithi162.stdout:1591 write 0x4bd5aed thru 0x4bd9793 (0x3ca7 bytes) 2024-09-16T11:13:42.340 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:42.341 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:42.344 INFO:teuthology.orchestra.run.smithi162.stdout:1592 write 0x5986baa thru 0x59875af (0xa06 bytes) 2024-09-16T11:13:42.351 INFO:teuthology.orchestra.run.smithi162.stdout:1593 read 0x72abdfd thru 0x72b96a3 (0xd8a7 bytes) 2024-09-16T11:13:42.351 INFO:teuthology.orchestra.run.smithi162.stdout:1594 read 0xe046560 thru 0xe047947 (0x13e8 bytes) 2024-09-16T11:13:42.351 INFO:teuthology.orchestra.run.smithi162.stdout:1595 read 0x9459b31 thru 0x945f77c (0x5c4c bytes) 2024-09-16T11:13:42.351 INFO:teuthology.orchestra.run.smithi162.stdout:1596 read 0x1cb3ff6 thru 0x1cbbb71 (0x7b7c bytes) 2024-09-16T11:13:42.351 INFO:teuthology.orchestra.run.smithi162.stdout:1598 read 0x4d068b9 thru 0x4d0fa6c (0x91b4 bytes) 2024-09-16T11:13:42.352 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:42.353 INFO:teuthology.orchestra.run.smithi162.stdout:1600 punch from 0xcb0c59a to 0xcb1b3f7, (0xee5d bytes) 2024-09-16T11:13:42.353 INFO:teuthology.orchestra.run.smithi162.stdout:1601 punch from 0x8bd608 to 0x8c106f, (0x3a67 bytes) 2024-09-16T11:13:42.353 INFO:teuthology.orchestra.run.smithi162.stdout:1602 read 0x10425bb thru 0x104a261 (0x7ca7 bytes) 2024-09-16T11:13:42.361 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:42.353+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:42.376 INFO:teuthology.orchestra.run.smithi162.stdout:1603 write 0x6c54db2 thru 0x6c64304 (0xf553 bytes) 2024-09-16T11:13:42.415 INFO:teuthology.orchestra.run.smithi162.stdout:1607 trunc from 0xe9417eb to 0xecfd588 2024-09-16T11:13:42.437 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:42.456 INFO:teuthology.orchestra.run.smithi162.stdout:1608 trunc from 0xecfd588 to 0x15b6b3d 2024-09-16T11:13:42.462 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:42.456+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:42.474 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:42.524 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:42.532 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:42.532 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:42.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:42.551 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:42.562 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:42.553+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:42.578 INFO:teuthology.orchestra.run.smithi162.stdout:1609 writesame 0xced8bb9 thru 0xcedfa78 data_size 0x10(0x6ec0 bytes) 2024-09-16T11:13:42.578 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:42.579 INFO:teuthology.orchestra.run.smithi162.stdout:1610 read 0x856a62e thru 0x8572b5d (0x8530 bytes) 2024-09-16T11:13:42.579 INFO:teuthology.orchestra.run.smithi162.stdout:1611 punch from 0x505c575 to 0x506a815, (0xe2a0 bytes) 2024-09-16T11:13:42.579 INFO:teuthology.orchestra.run.smithi162.stdout:1612 read 0x383e505 thru 0x383f93c (0x1438 bytes) 2024-09-16T11:13:42.579 INFO:teuthology.orchestra.run.smithi162.stdout:1614 write 0x2d521f6 thru 0x2d5df67 (0xbd72 bytes) 2024-09-16T11:13:42.599 INFO:teuthology.orchestra.run.smithi162.stdout:1615 trunc from 0xcedfa79 to 0x4d2fac9 2024-09-16T11:13:42.604 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:42.618 INFO:teuthology.orchestra.run.smithi162.stdout:1616 write 0x77dd683 thru 0x77dfb48 (0x24c6 bytes) 2024-09-16T11:13:42.626 INFO:teuthology.orchestra.run.smithi162.stdout:1617 write 0x716e8e5 thru 0x7173934 (0x5050 bytes) 2024-09-16T11:13:42.632 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:42.638 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:42.638 INFO:teuthology.orchestra.run.smithi162.stdout:1619 punch from 0x1e9e952 to 0x1ea3e38, (0x54e6 bytes) 2024-09-16T11:13:42.638 INFO:teuthology.orchestra.run.smithi162.stdout:1620 punch from 0x6305e4d to 0x630d6e1, (0x7894 bytes) 2024-09-16T11:13:42.638 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:42.638 INFO:teuthology.orchestra.run.smithi162.stdout:1622 read 0x23ba05b thru 0x23c0763 (0x6709 bytes) 2024-09-16T11:13:42.650 INFO:teuthology.orchestra.run.smithi162.stdout:1623 write 0xa00e048 thru 0xa01c1ad (0xe166 bytes) 2024-09-16T11:13:42.656 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:42.662 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:42.662 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:42.657+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:3875 4.17 4:ebebcad9:::rbd_data.11b5b93336d2.000000000000004b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.157719504999999, 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02646399, 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 3875 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503602+0000", 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503604+0000", 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503618+0000", 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.375e-05 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503642+0000", 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.357e-05 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503651+0000", 2024-09-16T11:13:42.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1060000000000006e-06 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503714+0000", 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3368999999999996e-05 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.503856+0000", 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014167699999999999 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.510013+0000", 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061567940000000002 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529819+0000", 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019806462 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.529932+0000", 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112724 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530010+0000", 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8412999999999996e-05 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:28.530066+0000", 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6004000000000001e-05 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.671 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:4158 4.0 4:047b769a:::rbd_data.11b5b93336d2.000000000000001d:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.554851256999999, 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.032637175999999997, 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 4158 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106471+0000", 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106472+0000", 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.35e-06 2024-09-16T11:13:42.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106479+0000", 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106496+0000", 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7578e-05 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106503+0000", 2024-09-16T11:13:42.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0899999999999999e-06 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106568+0000", 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4346999999999999e-05 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.106646+0000", 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8904000000000002e-05 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.114604+0000", 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0079574510000000008 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.138952+0000", 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024348172000000001 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139028+0000", 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6315000000000004e-05 2024-09-16T11:13:42.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139080+0000", 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1450999999999999e-05 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:29.139108+0000", 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7855e-05 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.677 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13707 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1129302939999999, 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023978039999999999, 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13707 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.678 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548392+0000", 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9299999999999997e-07 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548396+0000", 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8770000000000003e-06 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548405+0000", 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6130000000000004e-06 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548410+0000", 2024-09-16T11:13:42.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3639999999999998e-06 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.565053+0000", 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016642410999999999 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.565199+0000", 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014626400000000001 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.565740+0000", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054094600000000005 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.568994+0000", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032539610000000001 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572316+0000", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033225110000000002 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572342+0000", 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5338e-05 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572370+0000", 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7962000000000001e-05 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13708 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.548415+0000", 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1129065110000003, 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024336200999999998, 2024-09-16T11:13:42.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13708 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548415+0000", 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548415+0000", 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548416+0000", 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0699999999999996e-07 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548420+0000", 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548428+0000", 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0160000000000008e-06 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548433+0000", 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3530000000000002e-06 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.566374+0000", 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017940626000000001 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.566466+0000", 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1642000000000001e-05 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.566854+0000", 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00038811 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.569000+0000", 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021461100000000001 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572655+0000", 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003655265 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572716+0000", 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0887000000000002e-05 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572752+0000", 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5465e-05 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.689 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13709 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.1128832180000003, 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024492159999999999, 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13709 2024-09-16T11:13:42.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548439+0000", 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8000000000000005e-07 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548449+0000", 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8730000000000002e-06 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548477+0000", 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7846000000000001e-05 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.548498+0000", 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0590000000000001e-05 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.567398+0000", 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01890004 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.567519+0000", 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120987 2024-09-16T11:13:42.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.568010+0000", 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049080499999999997 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.569148+0000", 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011385740000000001 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572809+0000", 2024-09-16T11:13:42.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036606920000000001 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572869+0000", 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0239999999999999e-05 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.572931+0000", 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1733e-05 2024-09-16T11:13:42.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13798 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652642+0000", 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.0086797290000007, 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024192179000000001, 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13798 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652642+0000", 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652642+0000", 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652643+0000", 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3499999999999996e-07 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652646+0000", 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5750000000000001e-06 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652662+0000", 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5675000000000002e-05 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652666+0000", 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1930000000000003e-06 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.671029+0000", 2024-09-16T11:13:42.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018363180999999999 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.671142+0000", 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113064 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.671588+0000", 2024-09-16T11:13:42.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044516899999999998 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676658+0000", 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0050707720000000003 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676740+0000", 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1626999999999996e-05 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676789+0000", 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9224999999999997e-05 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676834+0000", 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5062999999999999e-05 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13799 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652698+0000", 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.0086234269999999, 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.024592899000000001, 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13799 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652698+0000", 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652698+0000", 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652699+0000", 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8899999999999998e-07 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652703+0000", 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8419999999999998e-06 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652710+0000", 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6640000000000001e-06 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652714+0000", 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9129999999999996e-06 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.673010+0000", 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020296209999999999 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.673088+0000", 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8663000000000002e-05 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.673463+0000", 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037484900000000002 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676670+0000", 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032069279999999999 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677216+0000", 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054598699999999995 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677258+0000", 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2052000000000003e-05 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677291+0000", 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3002e-05 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13800 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652735+0000", 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.0085867139999998, 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025687231000000001, 2024-09-16T11:13:42.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13800 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652735+0000", 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652735+0000", 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652736+0000", 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1500000000000004e-07 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652739+0000", 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.625e-06 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652746+0000", 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3810000000000004e-06 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652750+0000", 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9369999999999997e-06 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.674187+0000", 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021437060000000001 2024-09-16T11:13:42.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.674287+0000", 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100191 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.674772+0000", 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048459700000000002 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676681+0000", 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0019092410000000001 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678229+0000", 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001548694 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678350+0000", 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012061999999999999 2024-09-16T11:13:42.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678422+0000", 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2269999999999998e-05 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13801 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.652787+0000", 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.0085348090000004, 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025949692, 2024-09-16T11:13:42.715 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13801 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652787+0000", 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652787+0000", 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652788+0000", 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3200000000000004e-07 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652792+0000", 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9530000000000001e-06 2024-09-16T11:13:42.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652798+0000", 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6850000000000004e-06 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.652802+0000", 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7570000000000002e-06 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.675553+0000", 2024-09-16T11:13:42.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022751141999999999 2024-09-16T11:13:42.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.675663+0000", 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109462 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.676011+0000", 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034868900000000001 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.677038+0000", 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010261230000000001 2024-09-16T11:13:42.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678604+0000", 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001566663 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678688+0000", 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3770999999999995e-05 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.678737+0000", 2024-09-16T11:13:42.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8714999999999997e-05 2024-09-16T11:13:42.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13913 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.793159+0000", 2024-09-16T11:13:42.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.8681625050000008, 2024-09-16T11:13:42.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023814730999999999, 2024-09-16T11:13:42.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13913 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793159+0000", 2024-09-16T11:13:42.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793159+0000", 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793160+0000", 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2799999999999995e-07 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793165+0000", 2024-09-16T11:13:42.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2900000000000002e-06 2024-09-16T11:13:42.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793177+0000", 2024-09-16T11:13:42.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1963999999999999e-05 2024-09-16T11:13:42.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793185+0000", 2024-09-16T11:13:42.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4499999999999998e-06 2024-09-16T11:13:42.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.811958+0000", 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018772825999999999 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.812099+0000", 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000141643 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.812557+0000", 2024-09-16T11:13:42.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045780199999999999 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.815840+0000", 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0032831570000000001 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816895+0000", 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001054227 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816937+0000", 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2107000000000002e-05 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816974+0000", 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7437000000000001e-05 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:13914 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.733 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.793192+0000", 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.8681294130000001, 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023917087, 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 13914 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793192+0000", 2024-09-16T11:13:42.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793192+0000", 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793193+0000", 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4200000000000005e-07 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793198+0000", 2024-09-16T11:13:42.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1599999999999997e-06 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793212+0000", 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.394e-05 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.793220+0000", 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9529999999999999e-06 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.813913+0000", 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020693096000000001 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.814043+0000", 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000129228 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.814628+0000", 2024-09-16T11:13:42.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058514299999999999 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.816990+0000", 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023619549999999998 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.817035+0000", 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789000000000001e-05 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.817074+0000", 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9124e-05 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.817109+0000", 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5856999999999999e-05 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14022 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.739 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915633+0000", 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7456887220000006, 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023566409999999999, 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14022 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915633+0000", 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915633+0000", 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915634+0000", 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1699999999999997e-07 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915639+0000", 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2220000000000001e-06 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915652+0000", 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2815e-05 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915660+0000", 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0299999999999994e-06 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.931984+0000", 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.016323431999999999 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.932097+0000", 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113096 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.932626+0000", 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052935000000000005 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.937477+0000", 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0048505900000000001 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939008+0000", 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015318350000000001 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939118+0000", 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110041 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939200+0000", 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1081999999999994e-05 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.745 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14023 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915669+0000", 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7456531660000003, 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023738168, 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14023 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915669+0000", 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915669+0000", 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915670+0000", 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6700000000000002e-07 2024-09-16T11:13:42.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915675+0000", 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1429999999999997e-06 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915687+0000", 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2367e-05 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915698+0000", 2024-09-16T11:13:42.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1131999999999999e-05 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.933458+0000", 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.017759792 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.933514+0000", 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5896000000000002e-05 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.933789+0000", 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027467799999999999 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939230+0000", 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0054411609999999999 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939294+0000", 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3927e-05 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939359+0000", 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5839000000000004e-05 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939407+0000", 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7366e-05 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14024 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915707+0000", 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7456152809999992, 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023850314000000001, 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14024 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915707+0000", 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915707+0000", 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915708+0000", 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5999999999999991e-07 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915713+0000", 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915729+0000", 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5265e-05 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915736+0000", 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8480000000000001e-06 2024-09-16T11:13:42.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.934224+0000", 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.018487665 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.934302+0000", 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7494e-05 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.934651+0000", 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034901600000000001 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939248+0000", 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0045969330000000001 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939496+0000", 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024808700000000003 2024-09-16T11:13:42.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939530+0000", 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4532000000000001e-05 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939557+0000", 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6662999999999999e-05 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14025 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915744+0000", 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7455773600000004, 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023955583999999999, 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14025 2024-09-16T11:13:42.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915744+0000", 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915744+0000", 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915745+0000", 2024-09-16T11:13:42.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.09e-07 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915750+0000", 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.763e-06 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915762+0000", 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1498000000000001e-05 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915769+0000", 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8739999999999995e-06 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.935598+0000", 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019828124999999999 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.935733+0000", 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000135809 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.936414+0000", 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068093099999999996 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939569+0000", 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031548890000000001 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939609+0000", 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0225000000000002e-05 2024-09-16T11:13:42.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939650+0000", 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0500000000000002e-05 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939700+0000", 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0161000000000003e-05 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14026 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.915778+0000", 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7455436780000007, 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025917228, 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14026 2024-09-16T11:13:42.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915778+0000", 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915778+0000", 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915779+0000", 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915785+0000", 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:13:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915798+0000", 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3123e-05 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.915805+0000", 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2690000000000002e-06 2024-09-16T11:13:42.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.937533+0000", 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021727847000000002 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.937657+0000", 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012432099999999999 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.938233+0000", 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057565700000000003 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.939892+0000", 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016591939999999999 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.941608+0000", 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0017153520000000001 2024-09-16T11:13:42.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.941661+0000", 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3585000000000003e-05 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.941695+0000", 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4088000000000003e-05 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14060 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.960478+0000", 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7008437999999995, 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.023397426999999998, 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14060 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960478+0000", 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960478+0000", 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960479+0000", 2024-09-16T11:13:42.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9699999999999996e-07 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960482+0000", 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9299999999999999e-06 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960494+0000", 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2826999999999999e-05 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960507+0000", 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2875e-05 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.979744+0000", 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.019236559 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.979810+0000", 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6209000000000003e-05 2024-09-16T11:13:42.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.980108+0000", 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029817999999999999 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983011+0000", 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029025710000000001 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983742+0000", 2024-09-16T11:13:42.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073078499999999996 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983808+0000", 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6161000000000006e-05 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983875+0000", 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7732999999999997e-05 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14061 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.960512+0000", 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7008095010000002, 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025837055000000001, 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.777 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14061 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960512+0000", 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960512+0000", 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960513+0000", 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999995e-07 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960516+0000", 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3299999999999999e-06 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960524+0000", 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5000000000000002e-06 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960528+0000", 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.33e-06 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.981187+0000", 2024-09-16T11:13:42.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.020658826000000002 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.981286+0000", 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8914999999999997e-05 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.981569+0000", 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028276100000000003 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.983719+0000", 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021504060000000001 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986228+0000", 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002508906 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986298+0000", 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9985000000000002e-05 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986349+0000", 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1508999999999998e-05 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14062 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:33.960534+0000", 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.7007876109999991, 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025912858, 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.783 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14062 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960534+0000", 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960534+0000", 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960535+0000", 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960539+0000", 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.754e-06 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960560+0000", 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1228000000000001e-05 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.960575+0000", 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5585000000000001e-05 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.982114+0000", 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021538976000000001 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.982184+0000", 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9476999999999999e-05 2024-09-16T11:13:42.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.982497+0000", 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031305200000000002 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.984011+0000", 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015143489999999999 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986363+0000", 2024-09-16T11:13:42.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002351924 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986409+0000", 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6195000000000001e-05 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:33.986447+0000", 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7775999999999999e-05 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:14159 4.17 4:e97a8d37:::rbd_object_map.11b52f8117af:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:34.082343+0000", 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 8.5789790440000004, 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.022780277000000002, 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 14159 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:42.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082343+0000", 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082343+0000", 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082344+0000", 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082351+0000", 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.703e-06 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082366+0000", 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5760000000000002e-05 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.082377+0000", 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0519999999999999e-05 2024-09-16T11:13:42.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.101551+0000", 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.01917367 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.101608+0000", 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7265000000000001e-05 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.101836+0000", 2024-09-16T11:13:42.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000227798 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.104258+0000", 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024227659999999998 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.105028+0000", 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00076920000000000005 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.105085+0000", 2024-09-16T11:13:42.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6993000000000001e-05 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:34.105123+0000", 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8479000000000002e-05 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:42.795 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:42.797 INFO:teuthology.orchestra.run.smithi162.stdout:1625 writesame 0xb75450b thru 0xb75bc25 data_size 0x1(0x771b bytes) 2024-09-16T11:13:42.797 INFO:teuthology.orchestra.run.smithi162.stdout:1626 read 0x1472ca1 thru 0x147fb57 (0xceb7 bytes) 2024-09-16T11:13:42.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1627 read 0x479d4c1 thru 0x47a8b97 (0xb6d7 bytes) 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1628 punch from 0x54947d0 to 0x54966f5, (0x1f25 bytes) 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1629 write 0xbcfd5cf thru 0xbcffb75 (0x25a7 bytes) 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1631 read 0x6fb022e thru 0x6fb7fa1 (0x7d74 bytes) 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1632 writesame 0x97180e1 thru 0x971fd89 data_size 0x7(0x7ca9 bytes) 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1633 punch from 0xe49f86 to 0xe4f0c2, (0x513c bytes) 2024-09-16T11:13:42.799 INFO:teuthology.orchestra.run.smithi162.stdout:1634 read 0x653683f thru 0x6537f6c (0x172e bytes) 2024-09-16T11:13:42.800 INFO:teuthology.orchestra.run.smithi162.stdout:1636 read 0x57d130 thru 0x580f05 (0x3dd6 bytes) 2024-09-16T11:13:42.800 INFO:teuthology.orchestra.run.smithi162.stdout:1637 writesame 0xeb2811b thru 0xeb30578 data_size 0x1(0x845e bytes) 2024-09-16T11:13:42.800 INFO:teuthology.orchestra.run.smithi162.stdout:1639 clone 12 order 21 su 32768 sc 13 2024-09-16T11:13:42.800 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:42.789+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:42.897 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:42.889+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:42.907 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:42.907 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:42.917 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:42.998 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:42.992+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:43.017 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:43.017 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:43.017 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.026 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:43.099 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:43.092+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:43.129 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.129 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:43.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:4001 4.1b 4:dba71535:::rbd_data.11b5b93336d2.0000000000000016:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.37322163, 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.027255873, 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 4001 2024-09-16T11:13:43.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751420+0000", 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751422+0000", 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2469999999999998e-06 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751434+0000", 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1761e-05 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751469+0000", 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5577000000000002e-05 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751483+0000", 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3570999999999999e-05 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751628+0000", 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014536400000000001 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.751857+0000", 2024-09-16T11:13:43.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000228452 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.758485+0000", 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066286000000000001 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778412+0000", 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019927027 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778513+0000", 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010084 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778611+0000", 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8432999999999998e-05 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:28.778675+0000", 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4000999999999999e-05 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15969 4.d 4:b69afd92:::rbd_data.11b58a761a59.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:43.137 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.0455509440000004, 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044609673000000002, 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15969 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079092+0000", 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.17e-06 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079104+0000", 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079140+0000", 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5648000000000002e-05 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079152+0000", 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2636999999999999e-05 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079270+0000", 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011812200000000001 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079499+0000", 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022818 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.082238+0000", 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027396949999999999 2024-09-16T11:13:43.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123504+0000", 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041265556000000002 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123526+0000", 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1971999999999999e-05 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123623+0000", 2024-09-16T11:13:43.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7347000000000002e-05 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123700+0000", 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6786000000000006e-05 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15970 4.d 4:b7b5bf19:::rbd_data.11b58a761a59.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:43.143 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.0452462820000008, 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044492102999999998, 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15970 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079397+0000", 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.793e-06 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079405+0000", 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079421+0000", 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6201e-05 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079429+0000", 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.092788+0000", 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013358761 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.093013+0000", 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000225687 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.098959+0000", 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059459059999999999 2024-09-16T11:13:43.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123735+0000", 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024775755999999999 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123753+0000", 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8065999999999999e-05 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123836+0000", 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-05 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123887+0000", 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0559000000000002e-05 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:15969 4.d e47/36 ondisk, result = 0)", 2024-09-16T11:13:43.149 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.0347296139999997, 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033797389999999997, 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:43.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089913+0000", 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.04e-06 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089921+0000", 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2309999999999996e-06 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089941+0000", 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0494000000000001e-05 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089948+0000", 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4779999999999996e-06 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123470+0000", 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033522226000000002 2024-09-16T11:13:43.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123488+0000", 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8269000000000001e-05 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123709+0000", 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220652 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26656 4.9 4:95b731cd:::rbd_data.11b5499bda95.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.1737412760000003, 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.054121295999999999, 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26656 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950902+0000", 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950914+0000", 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1724e-05 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950943+0000", 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9621e-05 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950956+0000", 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.238e-05 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951090+0000", 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013472399999999999 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951475+0000", 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000384378 2024-09-16T11:13:43.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.964699+0000", 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013224217999999999 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004806+0000", 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.040106941 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004820+0000", 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3909e-05 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004895+0000", 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4905000000000002e-05 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.005021+0000", 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012639300000000001 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.159 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39149 4.c 4:37054e3a:::rbd_data.11b570fb0cd.0000000000000002:e [read 765952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.710869+0000", 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41377201899999999, 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022651594000000001, 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.160 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39149 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710869+0000", 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710869+0000", 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710870+0000", 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.004e-06 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710875+0000", 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9200000000000003e-06 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710928+0000", 2024-09-16T11:13:43.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2536e-05 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710963+0000", 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5410999999999997e-05 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733298+0000", 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022335229000000002 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733381+0000", 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2439000000000001e-05 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733521+0000", 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014005500000000001 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39150 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.164 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.710988+0000", 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41365345100000001, 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022586893, 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39150 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710988+0000", 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710988+0000", 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710989+0000", 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.697e-06 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.710995+0000", 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8200000000000002e-06 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711028+0000", 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3120000000000001e-05 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711052+0000", 2024-09-16T11:13:43.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3819999999999999e-05 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733560+0000", 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022507913000000001 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733575+0000", 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4523e-05 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39151 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711076+0000", 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41356505399999999, 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022548696999999999, 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39151 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711076+0000", 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711076+0000", 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711077+0000", 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.327e-06 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711083+0000", 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5940000000000001e-06 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711100+0000", 2024-09-16T11:13:43.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7306e-05 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711116+0000", 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6116999999999998e-05 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733599+0000", 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022482849999999999 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733625+0000", 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5503e-05 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39152 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711132+0000", 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.413509503, 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022523850000000002, 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39152 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711132+0000", 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711132+0000", 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711133+0000", 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1030000000000001e-06 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711149+0000", 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6252999999999998e-05 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711180+0000", 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0494e-05 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711193+0000", 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3104e-05 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733645+0000", 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022452367000000001 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733656+0000", 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0529e-05 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39155 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711258+0000", 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41338327899999999, 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022522983999999999, 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39155 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711258+0000", 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711258+0000", 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711259+0000", 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3e-07 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711262+0000", 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9569999999999998e-06 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711270+0000", 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2970000000000004e-06 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711275+0000", 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1100000000000002e-06 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733775+0000", 2024-09-16T11:13:43.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022499806000000001 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733781+0000", 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0839999999999997e-06 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39156 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711280+0000", 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41336078900000001, 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022537973999999999, 2024-09-16T11:13:43.181 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39156 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711280+0000", 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711280+0000", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711281+0000", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5999999999999991e-07 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711285+0000", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.337e-06 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711293+0000", 2024-09-16T11:13:43.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1000000000000004e-06 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711298+0000", 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9470000000000002e-06 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733808+0000", 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022510282 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733818+0000", 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0348e-05 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39157 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711304+0000", 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41333769799999998, 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022590934999999999, 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39157 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711304+0000", 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711304+0000", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711304+0000", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3e-07 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711307+0000", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9469999999999999e-06 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711315+0000", 2024-09-16T11:13:43.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9060000000000002e-06 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711322+0000", 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8340000000000001e-06 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733871+0000", 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.02254929 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733894+0000", 2024-09-16T11:13:43.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3227999999999999e-05 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39158 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711328+0000", 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.413313608, 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022610796999999998, 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.189 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39158 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711328+0000", 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711328+0000", 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711328+0000", 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9699999999999995e-07 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711331+0000", 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.773e-06 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711339+0000", 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9200000000000004e-06 2024-09-16T11:13:43.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711356+0000", 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6860000000000001e-05 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733932+0000", 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022575873 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733938+0000", 2024-09-16T11:13:43.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.674e-06 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39159 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711362+0000", 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41327903100000002, 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022609161999999999, 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.193 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39159 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711362+0000", 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711362+0000", 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711363+0000", 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6300000000000004e-07 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711366+0000", 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.88e-06 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711374+0000", 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1969999999999996e-06 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711379+0000", 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733961+0000", 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022581607 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.733971+0000", 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0662000000000001e-05 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39160 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711386+0000", 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41325509399999999, 2024-09-16T11:13:43.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022644745000000001, 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39160 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711386+0000", 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711386+0000", 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711387+0000", 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6599999999999995e-07 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711390+0000", 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.88e-06 2024-09-16T11:13:43.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711409+0000", 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9247000000000001e-05 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711419+0000", 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0034000000000001e-05 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734023+0000", 2024-09-16T11:13:43.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.02260388 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734031+0000", 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9379999999999992e-06 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39161 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711456+0000", 2024-09-16T11:13:43.201 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.413185527, 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022618172999999998, 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39161 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711456+0000", 2024-09-16T11:13:43.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711456+0000", 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711457+0000", 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.48e-06 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711462+0000", 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0329999999999999e-06 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711478+0000", 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5866999999999999e-05 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711488+0000", 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7399999999999999e-06 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734060+0000", 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022571783000000002 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734074+0000", 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4270000000000001e-05 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39162 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.205 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711522+0000", 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41311964299999998, 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022600648000000001, 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39162 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711522+0000", 2024-09-16T11:13:43.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711522+0000", 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711523+0000", 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.624e-06 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711528+0000", 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3029999999999999e-06 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711547+0000", 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8597000000000002e-05 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711570+0000", 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3102999999999999e-05 2024-09-16T11:13:43.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734111+0000", 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022540469 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734122+0000", 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1552000000000001e-05 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39163 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711596+0000", 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41304516600000002, 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022572608000000001, 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39163 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711596+0000", 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711596+0000", 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711597+0000", 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0669999999999999e-06 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711603+0000", 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4229999999999996e-06 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711621+0000", 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8807000000000001e-05 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711644+0000", 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2606999999999999e-05 2024-09-16T11:13:43.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734156+0000", 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022512069999999999 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734169+0000", 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2634e-05 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39164 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711657+0000", 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41298387800000003, 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022545863999999999, 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39164 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711657+0000", 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711657+0000", 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711658+0000", 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0100000000000001e-06 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711673+0000", 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4956e-05 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711703+0000", 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0156999999999999e-05 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711726+0000", 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2481e-05 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734191+0000", 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022465560999999998 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734203+0000", 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1698999999999999e-05 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:39165 4.c 4.5c72a0ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-16T11:13:43.217 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:42.711740+0000", 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.41290141400000002, 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022506061000000001, 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 39165 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711740+0000", 2024-09-16T11:13:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711740+0000", 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711741+0000", 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3570000000000001e-06 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711747+0000", 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711776+0000", 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9807000000000001e-05 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.711810+0000", 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4003e-05 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734232+0000", 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022421462 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:42.734246+0000", 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3939e-05 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.221 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:43.223 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:43.216+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:43.321 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:13:43.323 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:43.317+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:43.329 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:43.329 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:43.329 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.338 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:43.416 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone11 from 0xeb30579 (overlap 0x15b6b3d) to 0x8af7b2 2024-09-16T11:13:43.424 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:43.416+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:43.443 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:43.444 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:43.444 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.453 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:43.525 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:43.520+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:43.556 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.556 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:43.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:43.556 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:43.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.556 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:615 4.15 4:ad422a37:::rbd_data.11b52605a025.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 9437184] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.239514546999999, 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010270354000000001, 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 615 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311914+0000", 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311915+0000", 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1969999999999999e-06 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311920+0000", 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0010000000000001e-06 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311933+0000", 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2948e-05 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311937+0000", 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8219999999999997e-06 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311999+0000", 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1174999999999996e-05 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312158+0000", 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015924900000000001 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312292+0000", 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013398899999999999 2024-09-16T11:13:43.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313867+0000", 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001575087 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322101+0000", 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0082335659999999995 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322144+0000", 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3900000000000003e-05 2024-09-16T11:13:43.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322184+0000", 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9419999999999999e-05 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 22.688474298999999, 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:13:43.562 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:13:43.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2068 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.026950514999999, 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099591620000000006, 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2068 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524478+0000", 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524479+0000", 2024-09-16T11:13:43.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.155e-06 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524483+0000", 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5680000000000001e-06 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524497+0000", 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4018e-05 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524509+0000", 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1735e-05 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534308+0000", 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097988120000000005 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534360+0000", 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2094e-05 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534437+0000", 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6779999999999993e-05 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2069 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.026902528000001, 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010200859, 2024-09-16T11:13:43.571 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2069 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524526+0000", 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524527+0000", 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.02e-06 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524531+0000", 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4900000000000002e-06 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524544+0000", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.311e-05 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.524552+0000", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6879999999999999e-06 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534481+0000", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099292119999999998 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534629+0000", 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148281 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534726+0000", 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7058000000000005e-05 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2082 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2228224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.026355424999998, 2024-09-16T11:13:43.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010054116, 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2082 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525073+0000", 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525074+0000", 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4470000000000001e-06 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525080+0000", 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8980000000000001e-06 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525099+0000", 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8960000000000001e-05 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525112+0000", 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3013999999999999e-05 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.534937+0000", 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098250449999999993 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535013+0000", 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6108999999999999e-05 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535127+0000", 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113643 2024-09-16T11:13:43.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2083 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.026301596, 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010171747, 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.580 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2083 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525126+0000", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525128+0000", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.082e-06 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525134+0000", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8079999999999999e-06 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525149+0000", 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4833e-05 2024-09-16T11:13:43.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525158+0000", 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5820000000000002e-06 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535171+0000", 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010012828999999999 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535224+0000", 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3826999999999998e-05 2024-09-16T11:13:43.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535298+0000", 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3786000000000001e-05 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2098 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.026189665, 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010231951, 2024-09-16T11:13:43.584 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2098 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525238+0000", 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525240+0000", 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.494e-06 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525247+0000", 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7710000000000001e-06 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525265+0000", 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7966e-05 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525276+0000", 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1827e-05 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535332+0000", 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010055633 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535388+0000", 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5741e-05 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535470+0000", 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2519000000000001e-05 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2099 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.025975602999999, 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010173994, 2024-09-16T11:13:43.588 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2099 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525452+0000", 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525455+0000", 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6469999999999998e-06 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525468+0000", 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2931e-05 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525518+0000", 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0448000000000002e-05 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525542+0000", 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3870999999999999e-05 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535510+0000", 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099681400000000003 2024-09-16T11:13:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535559+0000", 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8028999999999997e-05 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535626+0000", 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7928000000000004e-05 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2100 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.025867372, 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010216233999999999, 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2100 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525561+0000", 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525562+0000", 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3030000000000001e-06 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525569+0000", 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9240000000000001e-06 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525596+0000", 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7430999999999999e-05 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525645+0000", 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8597000000000002e-05 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535663+0000", 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010018132000000001 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535711+0000", 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7706e-05 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535777+0000", 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6141000000000003e-05 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:2101 4.8 4:155f72ae:::rbd_data.11b51c0df580.0000000000000001:7 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.025767908999999, 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010279129999999999, 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 2101 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525660+0000", 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525662+0000", 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.437e-06 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525668+0000", 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0900000000000001e-06 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525709+0000", 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1464000000000001e-05 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.525726+0000", 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7210999999999998e-05 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535807+0000", 2024-09-16T11:13:43.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010080621999999999 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535872+0000", 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5504000000000002e-05 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:24.535939+0000", 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6802000000000002e-05 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.601 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22419 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 163840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.406750+0000", 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1446775679999996, 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010174645, 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22419 2024-09-16T11:13:43.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406750+0000", 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406750+0000", 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406752+0000", 2024-09-16T11:13:43.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1930000000000001e-06 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406757+0000", 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.57e-06 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406771+0000", 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3942e-05 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406779+0000", 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.52e-06 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.416695+0000", 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099158579999999996 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.416786+0000", 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1114999999999998e-05 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.416925+0000", 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013944700000000001 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22429 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.406788+0000", 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1446401250000005, 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010335486, 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22429 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406788+0000", 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406788+0000", 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406789+0000", 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0279999999999999e-06 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406793+0000", 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2150000000000003e-06 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406807+0000", 2024-09-16T11:13:43.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4083000000000001e-05 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406815+0000", 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9230000000000002e-06 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.416966+0000", 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01015126 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417048+0000", 2024-09-16T11:13:43.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1656000000000006e-05 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417123+0000", 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5321000000000006e-05 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22443 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.406824+0000", 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1446041769999997, 2024-09-16T11:13:43.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010621429, 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22443 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406824+0000", 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406824+0000", 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406825+0000", 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9800000000000002e-07 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406829+0000", 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:13:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406842+0000", 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3443e-05 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406850+0000", 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8990000000000001e-06 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417163+0000", 2024-09-16T11:13:43.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010313566 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417301+0000", 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000137813 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417445+0000", 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014401 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22461 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 155648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.406858+0000", 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1445697910000003, 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010979203, 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22461 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406858+0000", 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406858+0000", 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406859+0000", 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0500000000000002e-07 2024-09-16T11:13:43.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406864+0000", 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6129999999999997e-06 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406877+0000", 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3492000000000001e-05 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406885+0000", 2024-09-16T11:13:43.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1000000000000004e-06 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417505+0000", 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010619679999999999 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417628+0000", 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012255100000000001 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417837+0000", 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000209862 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22467 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.406894+0000", 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1445341630000003, 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011221475, 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22467 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406894+0000", 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406894+0000", 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406895+0000", 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8800000000000001e-07 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.406912+0000", 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7317e-05 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408279+0000", 2024-09-16T11:13:43.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001367035 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408326+0000", 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6539999999999998e-05 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.417888+0000", 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095620830000000007 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418000+0000", 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011271 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418115+0000", 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011490200000000001 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22493 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.408343+0000", 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1430846150000002, 2024-09-16T11:13:43.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010008638, 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22493 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408343+0000", 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408343+0000", 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408345+0000", 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.55e-06 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408352+0000", 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3529999999999997e-06 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408370+0000", 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7963000000000001e-05 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408381+0000", 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0261999999999999e-05 2024-09-16T11:13:43.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418143+0000", 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097622909999999993 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418223+0000", 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0422999999999999e-05 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418352+0000", 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000128796 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22499 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 172032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.408391+0000", 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1430373039999999, 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010165581999999999, 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.628 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22499 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408391+0000", 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408391+0000", 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408392+0000", 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.455e-06 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408397+0000", 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5870000000000003e-06 2024-09-16T11:13:43.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408411+0000", 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3798000000000001e-05 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408419+0000", 2024-09-16T11:13:43.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6519999999999995e-06 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418380+0000", 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099606859999999998 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418451+0000", 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1203000000000003e-05 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418556+0000", 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000105201 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22517 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 221184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.408602+0000", 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1428264559999999, 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010165151000000001, 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22517 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408602+0000", 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408602+0000", 2024-09-16T11:13:43.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408603+0000", 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.204e-06 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408607+0000", 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5560000000000001e-06 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408624+0000", 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6160000000000001e-05 2024-09-16T11:13:43.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408632+0000", 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6249999999999996e-06 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418592+0000", 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0099602100000000006 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418667+0000", 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4283999999999996e-05 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418767+0000", 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010011200000000001 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.637 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22523 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.408641+0000", 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1427866609999997, 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010360996000000001, 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22523 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408641+0000", 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408641+0000", 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408642+0000", 2024-09-16T11:13:43.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6000000000000002e-07 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408646+0000", 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2100000000000003e-06 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408659+0000", 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.278e-05 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408667+0000", 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0059999999999992e-06 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418809+0000", 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010141996 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.418898+0000", 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8503000000000003e-05 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419002+0000", 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000104641 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22537 4.15 4:aa53788b:::rbd_data.11b52f8117af.0000000000000007:a [read 196608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.408717+0000", 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.1427107349999996, 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010508117000000001, 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.642 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22537 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408717+0000", 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408717+0000", 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408718+0000", 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.032e-06 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408733+0000", 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4318000000000001e-05 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408750+0000", 2024-09-16T11:13:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7686999999999999e-05 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.408759+0000", 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419039+0000", 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010280315999999999 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419128+0000", 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8814999999999996e-05 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419225+0000", 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7163000000000005e-05 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.646 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.647 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-16T11:13:43.648 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:43.649 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:43.640+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:43.749 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:43.741+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:43.754 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.754 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:43.754 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:43.754 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:43.850 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:43.844+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:43.861 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:43.861 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:43.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:43.951 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:43.945+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:603 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:43.976 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.668269873, 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019179324000000001, 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 603 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:43.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302823+0000", 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302824+0000", 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.06e-06 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302830+0000", 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9209999999999997e-06 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302844+0000", 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3304e-05 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.302850+0000", 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5660000000000003e-06 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310529+0000", 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0076792980000000002 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.310601+0000", 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1070999999999998e-05 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311059+0000", 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00045884099999999998 2024-09-16T11:13:43.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.313207+0000", 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002147805 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321862+0000", 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0086545330000000007 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.321919+0000", 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7130999999999999e-05 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322003+0000", 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3794000000000004e-05 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.982 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:604 4.a 4:57f43eeb:::rbd_object_map.11b52605a025:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 23.667463876999999, 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.018458927999999999, 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 604 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303629+0000", 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303631+0000", 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5579999999999999e-06 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303638+0000", 2024-09-16T11:13:43.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303654+0000", 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6628000000000001e-05 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.303662+0000", 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8390000000000007e-06 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311727+0000", 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080650300000000008 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.311786+0000", 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8507000000000001e-05 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.312144+0000", 2024-09-16T11:13:43.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358821 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.314152+0000", 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002008045 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322013+0000", 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0078606920000000007 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:43.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322053+0000", 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0055999999999997e-05 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.322088+0000", 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4972000000000003e-05 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22538 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 196608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.419986+0000", 2024-09-16T11:13:43.988 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5511068159999999, 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015947718, 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22538 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419986+0000", 2024-09-16T11:13:43.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419986+0000", 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419987+0000", 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419989+0000", 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2500000000000001e-06 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419995+0000", 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.419998+0000", 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.569e-06 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.435744+0000", 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015746118 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.435834+0000", 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9265000000000007e-05 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.435934+0000", 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100332 2024-09-16T11:13:43.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:22548 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 208896~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.420002+0000", 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5510908810000004, 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016163042999999998, 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 22548 2024-09-16T11:13:43.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.420002+0000", 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.420002+0000", 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.420003+0000", 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.420009+0000", 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6019999999999996e-06 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.420016+0000", 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3169999999999998e-06 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.420020+0000", 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0130000000000004e-06 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.435968+0000", 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015948666 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.436058+0000", 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9895999999999994e-05 2024-09-16T11:13:43.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.436165+0000", 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107168 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23130 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 335872~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441186+0000", 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5299069889999997, 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015939168, 2024-09-16T11:13:43.997 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23130 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441186+0000", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441186+0000", 2024-09-16T11:13:43.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441187+0000", 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.72e-07 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441188+0000", 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9080000000000002e-06 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:43.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441194+0000", 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441197+0000", 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4910000000000001e-06 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.456778+0000", 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015580480000000001 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.456848+0000", 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0668999999999997e-05 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457125+0000", 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00027687099999999999 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23144 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 385024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.001 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441202+0000", 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5298908999999998, 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016126262999999998, 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23144 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441202+0000", 2024-09-16T11:13:44.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441202+0000", 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441203+0000", 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2800000000000003e-07 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441205+0000", 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1110000000000002e-06 2024-09-16T11:13:44.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441211+0000", 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.852e-06 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441214+0000", 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7620000000000001e-06 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457173+0000", 2024-09-16T11:13:44.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015958569999999998 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457245+0000", 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2362999999999998e-05 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457329+0000", 2024-09-16T11:13:44.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3276999999999996e-05 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23149 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 348160~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441218+0000", 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5298752660000003, 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016335532999999999, 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23149 2024-09-16T11:13:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441218+0000", 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441218+0000", 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441218+0000", 2024-09-16T11:13:44.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.72e-07 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441221+0000", 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3860000000000001e-06 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441226+0000", 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441229+0000", 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3530000000000002e-06 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457359+0000", 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016130261999999999 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457443+0000", 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3683000000000005e-05 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457553+0000", 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000110896 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23154 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 385024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441245+0000", 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5298482939999998, 2024-09-16T11:13:44.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016526156, 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23154 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441245+0000", 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441245+0000", 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441245+0000", 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3200000000000001e-07 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441251+0000", 2024-09-16T11:13:44.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6849999999999998e-06 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441262+0000", 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1172000000000001e-05 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441265+0000", 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000002e-06 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457585+0000", 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016320014000000001 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457672+0000", 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6965999999999994e-05 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457771+0000", 2024-09-16T11:13:44.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9216999999999996e-05 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23160 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 348160~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441268+0000", 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5298254470000003, 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016724139999999998, 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23160 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441268+0000", 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441268+0000", 2024-09-16T11:13:44.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441268+0000", 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441269+0000", 2024-09-16T11:13:44.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.516e-06 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441273+0000", 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2669999999999998e-06 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441275+0000", 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2019999999999998e-06 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457856+0000", 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016581366 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457910+0000", 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3857999999999998e-05 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.457992+0000", 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1691000000000001e-05 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.020 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23169 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 360448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441277+0000", 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5298162499999997, 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016964288000000001, 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.021 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23169 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441277+0000", 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441277+0000", 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441277+0000", 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441279+0000", 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4610000000000001e-06 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441282+0000", 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9749999999999999e-06 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441284+0000", 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458049+0000", 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01676505 2024-09-16T11:13:44.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458144+0000", 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5347999999999996e-05 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458241+0000", 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6769999999999997e-05 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23173 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 360448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441286+0000", 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5298070949999998, 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017126270999999998, 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23173 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441286+0000", 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441286+0000", 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441286+0000", 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4400000000000001e-07 2024-09-16T11:13:44.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441288+0000", 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4559999999999999e-06 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441291+0000", 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.072e-06 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441293+0000", 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4229999999999999e-06 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458278+0000", 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01698479 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458337+0000", 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8774999999999999e-05 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458412+0000", 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5511000000000003e-05 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23184 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 372736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441295+0000", 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5297977149999999, 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017249342000000001, 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.030 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23184 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441295+0000", 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441295+0000", 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441296+0000", 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4699999999999998e-07 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441297+0000", 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2580000000000001e-06 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441300+0000", 2024-09-16T11:13:44.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1109999999999999e-06 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441303+0000", 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9270000000000001e-06 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458428+0000", 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017124871 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458476+0000", 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7998999999999999e-05 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458545+0000", 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8929000000000004e-05 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23188 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 372736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.034 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.441305+0000", 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5297881350000004, 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017388104000000001, 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23188 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441305+0000", 2024-09-16T11:13:44.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441305+0000", 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441305+0000", 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.48e-07 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441306+0000", 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0640000000000001e-06 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441309+0000", 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.441311+0000", 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458578+0000", 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017266476999999999 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458621+0000", 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3503e-05 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.458693+0000", 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1716000000000003e-05 2024-09-16T11:13:44.038 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23217 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443473+0000", 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5276205320000003, 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015793333, 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.039 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23217 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443473+0000", 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443473+0000", 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443473+0000", 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443475+0000", 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4080000000000001e-06 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443481+0000", 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8660000000000002e-06 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443485+0000", 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1359999999999999e-06 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459160+0000", 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015674976 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459204+0000", 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3538999999999998e-05 2024-09-16T11:13:44.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459266+0000", 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2038999999999993e-05 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23222 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443490+0000", 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5276036240000002, 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015922902999999999, 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.043 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23222 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443490+0000", 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443490+0000", 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443490+0000", 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443506+0000", 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6280999999999999e-05 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443528+0000", 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2031999999999998e-05 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443533+0000", 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4479999999999996e-06 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459292+0000", 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015759428999999998 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459349+0000", 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6727000000000001e-05 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459412+0000", 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3578999999999996e-05 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.047 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23223 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443539+0000", 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5275542959999999, 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01602259, 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23223 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443539+0000", 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443539+0000", 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443540+0000", 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7700000000000004e-07 2024-09-16T11:13:44.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443543+0000", 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0479999999999999e-06 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443549+0000", 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443552+0000", 2024-09-16T11:13:44.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6519999999999999e-06 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459444+0000", 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015891729 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459488+0000", 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4319e-05 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459561+0000", 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3027000000000004e-05 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23227 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443560+0000", 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5275336470000003, 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016145504000000001, 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.052 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23227 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443560+0000", 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443560+0000", 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443560+0000", 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.53e-07 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443562+0000", 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.193e-06 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443568+0000", 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.942e-06 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443572+0000", 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530000000000001e-06 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459584+0000", 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016011721 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459640+0000", 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6543999999999999e-05 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459705+0000", 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4797999999999999e-05 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23239 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443576+0000", 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5275170129999998, 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016266867000000001, 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23239 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443576+0000", 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443576+0000", 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443576+0000", 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2099999999999998e-07 2024-09-16T11:13:44.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443579+0000", 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5840000000000002e-06 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443585+0000", 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1310000000000003e-06 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443588+0000", 2024-09-16T11:13:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2569999999999999e-06 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459735+0000", 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016146565000000002 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459779+0000", 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4407999999999999e-05 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459843+0000", 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3601000000000003e-05 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23241 4.a 4:540d3a91:::rbd_data.11b52f8117af.000000000000000c:a [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443592+0000", 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.5275009329999998, 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016408022000000001, 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.061 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23241 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443592+0000", 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443592+0000", 2024-09-16T11:13:44.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443593+0000", 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8599999999999999e-07 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443595+0000", 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.198e-06 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443601+0000", 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7389999999999996e-06 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443604+0000", 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5370000000000002e-06 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459878+0000", 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016273437000000002 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.459939+0000", 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1209000000000003e-05 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.460000+0000", 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1515999999999999e-05 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:23251 4.a 4:56442492:::rbd_data.11b52f8117af.0000000000000000:a [read 376832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.443608+0000", 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.527485392, 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016539221, 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 23251 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443608+0000", 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443608+0000", 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443608+0000", 2024-09-16T11:13:44.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443610+0000", 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2819999999999999e-06 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443616+0000", 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2650000000000004e-06 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.443619+0000", 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3900000000000002e-06 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.460032+0000", 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016412435999999999 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.460076+0000", 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4795000000000001e-05 2024-09-16T11:13:44.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.460147+0000", 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0674999999999997e-05 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.070 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:44.073 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:44.073 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:44.069+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:44.073 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:44.073 INFO:teuthology.orchestra.run.smithi162.stdout:1641 read 0x492e2be thru 0x493df66 (0xfca9 bytes) 2024-09-16T11:13:44.073 INFO:teuthology.orchestra.run.smithi162.stdout:1642 punch from 0x86adba to 0x87791e, (0xcb64 bytes) 2024-09-16T11:13:44.073 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:44.073 INFO:teuthology.orchestra.run.smithi162.stdout:1644 write 0xe902f56 thru 0xe908349 (0x53f4 bytes) 2024-09-16T11:13:44.074 INFO:teuthology.orchestra.run.smithi162.stdout:1646 punch from 0xb65967f to 0xb65d440, (0x3dc1 bytes) 2024-09-16T11:13:44.074 INFO:teuthology.orchestra.run.smithi162.stdout:1648 punch from 0x74ba3a6 to 0x74bbad1, (0x172b bytes) 2024-09-16T11:13:44.074 INFO:teuthology.orchestra.run.smithi162.stdout:1649 write 0x3b02f32 thru 0x3b0beaa (0x8f79 bytes) 2024-09-16T11:13:44.074 INFO:teuthology.orchestra.run.smithi162.stdout:1650 writesame 0x757978 thru 0x764a06 data_size 0x27(0xd08f bytes) 2024-09-16T11:13:44.116 INFO:teuthology.orchestra.run.smithi162.stdout:1651 read 0x1e827f6 thru 0x1e90968 (0xe173 bytes) 2024-09-16T11:13:44.143 INFO:teuthology.orchestra.run.smithi162.stdout:1652 trunc from 0xeb30579 to 0x13acc64 2024-09-16T11:13:44.173 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:44.165+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:44.195 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:44.196 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:44.196 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:44.196 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:44.207 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:44.273 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:44.268+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:44.300 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:44.300 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:44.300 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:44.300 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:44.300 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:44.309 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:44.368 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:44.375 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:44.369+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:44.395 INFO:teuthology.orchestra.run.smithi162.stdout:1657 write 0xb2156c5 thru 0xb21e356 (0x8c92 bytes) 2024-09-16T11:13:44.420 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.638067278000001, 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:44.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:44.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:44.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:44.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:44.428 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.250141577999997, 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:44.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:44.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:44.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.434 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249990248000003, 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:44.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249938761999999, 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:44.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249868614, 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.447 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:44.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:44.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249696143999998, 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.453 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:44.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:44.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249637821999997, 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.459 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:44.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:44.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249485702000001, 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.465 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:44.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:44.471 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249307182999999, 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:44.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:44.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.249274210000003, 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:44.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:44.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:44.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:44.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.697108718000003, 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:44.483 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:44.488 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.697086485, 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:44.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:44.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:44.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.696957687999998, 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:44.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:44.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.696899342999998, 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:44.501 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:44.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:44.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.696844953999999, 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:44.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:44.512 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.696710647000003, 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.513 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:44.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:44.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.518 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.915613127, 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.519 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:44.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:44.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.915555277999999, 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.525 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:44.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:44.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.915467966000001, 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.531 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:44.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:44.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.915229838000002, 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:44.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:44.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:44.542 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:44.544 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:44.537+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:44.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:44.545 INFO:teuthology.orchestra.run.smithi162.stdout:1658 writesame 0xee31a17 thru 0xee36a8b data_size 0x1(0x5075 bytes) 2024-09-16T11:13:44.545 INFO:teuthology.orchestra.run.smithi162.stdout:1659 read 0x15ae78d thru 0x15bd5c7 (0xee3b bytes) 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1661 trunc from 0xee36a8c to 0x81aeff7 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1663 writesame 0xe18a19d thru 0xe1940be data_size 0x1(0x9f22 bytes) 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1665 read 0x5b60e0d thru 0x5b65a2a (0x4c1e bytes) 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1667 read 0xd89c8dd thru 0xd8a022e (0x3952 bytes) 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1668 read 0x9e7f4 thru 0xa01f8 (0x1a05 bytes) 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1669 trunc from 0xe1940bf to 0x70545c2 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1670 writesame 0x99346bd thru 0x993ee2d data_size 0x1(0xa771 bytes) 2024-09-16T11:13:44.546 INFO:teuthology.orchestra.run.smithi162.stdout:1671 write 0x48fb856 thru 0x4906818 (0xafc3 bytes) 2024-09-16T11:13:44.558 INFO:teuthology.orchestra.run.smithi162.stdout:1672 write 0xdfd6107 thru 0xdfe2b5b (0xca55 bytes) 2024-09-16T11:13:44.569 INFO:teuthology.orchestra.run.smithi162.stdout:1673 read 0x46d967b thru 0x46e0dba (0x7740 bytes) 2024-09-16T11:13:44.569 INFO:teuthology.orchestra.run.smithi162.stdout:1674 write 0x130f626 thru 0x131c871 (0xd24c bytes) 2024-09-16T11:13:44.573 INFO:teuthology.orchestra.run.smithi162.stdout:1675 read 0x4d50002 thru 0x4d531e7 (0x31e6 bytes) 2024-09-16T11:13:44.573 INFO:teuthology.orchestra.run.smithi162.stdout:1676 writesame 0x93f7b57 thru 0x93ff08d data_size 0x1(0x7537 bytes) 2024-09-16T11:13:44.601 INFO:teuthology.orchestra.run.smithi162.stdout:1677 read 0x530e0c1 thru 0x530e150 (0x90 bytes) 2024-09-16T11:13:44.601 INFO:teuthology.orchestra.run.smithi162.stdout:1678 writesame 0x252610c thru 0x2528e0a data_size 0x1(0x2cff bytes) 2024-09-16T11:13:44.617 INFO:teuthology.orchestra.run.smithi162.stdout:1679 writesame 0xa356c84 thru 0xa35d848 data_size 0x1(0x6bc5 bytes) 2024-09-16T11:13:44.635 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:44.636 INFO:teuthology.orchestra.run.smithi162.stdout:1682 punch from 0xb91b37f to 0xb925e81, (0xab02 bytes) 2024-09-16T11:13:44.645 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:44.637+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:44.649 INFO:teuthology.orchestra.run.smithi162.stdout:1683 trunc from 0xdfe2b5c to 0x769869d 2024-09-16T11:13:44.652 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:44.652 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43956 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.649210+0000", 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.0023372419999999998, 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002400658, 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43956 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.653 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649210+0000", 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649210+0000", 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649213+0000", 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4370000000000001e-06 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649226+0000", 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2799000000000001e-05 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649252+0000", 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6533000000000002e-05 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649264+0000", 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1569999999999999e-05 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649366+0000", 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010235599999999999 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.649546+0000", 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017955299999999999 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.650177+0000", 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00063124600000000002 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:13:44.657 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:44.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:44.679 INFO:teuthology.orchestra.run.smithi162.stdout:1687 trunc from 0x769869d to 0x1cd6dbc 2024-09-16T11:13:44.708 INFO:teuthology.orchestra.run.smithi162.stdout:1688 trunc from 0x1cd6dbc to 0x473812b 2024-09-16T11:13:44.720 INFO:teuthology.orchestra.run.smithi162.stdout:1689 write 0x4692a1b thru 0x469bc3b (0x9221 bytes) 2024-09-16T11:13:44.745 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:44.740+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:44.754 INFO:teuthology.orchestra.run.smithi162.stdout:1690 write 0xb3f0c42 thru 0xb3f1a74 (0xe33 bytes) 2024-09-16T11:13:44.760 INFO:teuthology.orchestra.run.smithi162.stdout:1692 read 0x55a42b1 thru 0x55aa8c4 (0x6614 bytes) 2024-09-16T11:13:44.760 INFO:teuthology.orchestra.run.smithi162.stdout:1693 flatten 2024-09-16T11:13:44.773 INFO:teuthology.orchestra.run.smithi162.stdout:1694 punch from 0xc17519 to 0xc25d8e, (0xe875 bytes) 2024-09-16T11:13:44.773 INFO:teuthology.orchestra.run.smithi162.stdout:1695 read 0x83446d thru 0x83bbc4 (0x7758 bytes) 2024-09-16T11:13:44.774 INFO:teuthology.orchestra.run.smithi162.stdout:1696 clone 13 order 21 su 262144 sc 10 2024-09-16T11:13:44.783 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:44.783 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:44.783 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:44.784 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:44.784 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:44.793 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:44.798 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:44.846 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:44.837+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:44.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.76020935000000001, 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:13:44.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:13:44.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.76013160700000004, 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:13:44.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75776559200000004, 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:13:44.947 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:13:44.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:13:44.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75774607599999999, 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:13:44.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75660857199999998, 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:13:44.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:44.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75659267299999999, 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:44.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:13:44.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.964 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75656841900000005, 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:13:44.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:13:44.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75655315999999995, 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:13:44.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:13:44.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:13:44.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75653784099999999, 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:13:44.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:13:44.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:13:44.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:13:44.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75451543700000001, 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:13:44.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:13:44.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75450093900000004, 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:13:44.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75448660599999995, 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:13:44.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.995 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75447254900000005, 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:44.996 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:13:44.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:13:44.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:45.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75445869300000001, 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:45.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:13:45.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75444481799999996, 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.005 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:45.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75443105099999996, 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75438354200000002, 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:13:45.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:13:45.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:13:45.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.75409772399999997, 2024-09-16T11:13:45.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:13:45.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.022 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.66143390800000001, 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:13:45.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:45.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:45.029 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.66128472599999999, 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:45.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:13:45.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:13:45.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:45.035 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:45.037 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:45.038 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:45.038 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:45.038 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:45.038 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:45.038 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:45.028+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:45.052 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:45.073 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:45.101 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:45.125 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:45.128 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:45.137 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:45.129+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:45.147 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.147 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.157 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:45.238 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:45.232+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:45.257 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.257 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:45.332 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:13:45.339 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:45.333+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:45.367 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.368 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.377 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:45.439 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:45.432+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:45.474 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.474 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:45.503 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:45.504 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:45.504 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-16T11:13:45.504 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:13:45.504 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:13:45.504 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-16T11:13:45.541 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:45.537+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:45.600 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.600 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.610 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:45.619 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-16T11:13:45.629 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:45.629 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:13:45.629 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:45.641 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:45.636+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:45.742 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:45.733+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:45.747 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.747 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.761 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:45.843 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:45.841+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:45.849 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.849 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.859 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:45.939 INFO:teuthology.orchestra.run.smithi162.stdout:1697 punch from 0x6af6e88 to 0x6b00249, (0x93c1 bytes) 2024-09-16T11:13:45.944 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:45.937+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:45.945 INFO:teuthology.orchestra.run.smithi162.stdout:1699 write 0x44c388b thru 0x44cc2ae (0x8a24 bytes) 2024-09-16T11:13:45.953 INFO:teuthology.orchestra.run.smithi162.stdout:1700 read 0xad9a721 thru 0xad9dfd5 (0x38b5 bytes) 2024-09-16T11:13:45.953 INFO:teuthology.orchestra.run.smithi162.stdout:1701 write 0x99faf2e thru 0x99fb039 (0x10c bytes) 2024-09-16T11:13:45.963 INFO:teuthology.orchestra.run.smithi162.stdout:1702 write 0xa71824d thru 0xa71e7e6 (0x659a bytes) 2024-09-16T11:13:45.964 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:45.965 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:45.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:45.983 INFO:teuthology.orchestra.run.smithi162.stdout:1703 write 0xc0511d8 thru 0xc05161f (0x448 bytes) 2024-09-16T11:13:45.989 INFO:teuthology.orchestra.run.smithi162.stdout:1704 read 0xb158f7b thru 0xb15c630 (0x36b6 bytes) 2024-09-16T11:13:45.989 INFO:teuthology.orchestra.run.smithi162.stdout:1705 punch from 0xee4bb5 to 0xef30d2, (0xe51d bytes) 2024-09-16T11:13:45.989 INFO:teuthology.orchestra.run.smithi162.stdout:1707 write 0x423c672 thru 0x424330e (0x6c9d bytes) 2024-09-16T11:13:45.999 INFO:teuthology.orchestra.run.smithi162.stdout:1709 read 0x4c81018 thru 0x4c89676 (0x865f bytes) 2024-09-16T11:13:46.000 INFO:teuthology.orchestra.run.smithi162.stdout:1710 punch from 0xb9b25d3 to 0xb9b6a46, (0x4473 bytes) 2024-09-16T11:13:46.000 INFO:teuthology.orchestra.run.smithi162.stdout:1711 read 0xf45b5b thru 0xf4d0ba (0x7560 bytes) 2024-09-16T11:13:46.002 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:46.002 INFO:teuthology.orchestra.run.smithi162.stdout:1713 writesame 0x66889cc thru 0x668ed54 data_size 0x1(0x6389 bytes) 2024-09-16T11:13:46.026 INFO:teuthology.orchestra.run.smithi162.stdout:1714 writesame 0xb0b5ae3 thru 0xb0b902d data_size 0x1(0x354b bytes) 2024-09-16T11:13:46.042 INFO:teuthology.orchestra.run.smithi162.stdout:1716 read 0x154a652 thru 0x155a224 (0xfbd3 bytes) 2024-09-16T11:13:46.045 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:46.040+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:46.059 INFO:teuthology.orchestra.run.smithi162.stdout:1717 write 0xdc954a3 thru 0xdca17f8 (0xc356 bytes) 2024-09-16T11:13:46.063 INFO:teuthology.orchestra.run.smithi162.stdout:1718 read 0xbf773d1 thru 0xbf82ade (0xb70e bytes) 2024-09-16T11:13:46.068 INFO:teuthology.orchestra.run.smithi162.stdout:1719 trunc from 0xdca17f9 to 0xb2cac5f 2024-09-16T11:13:46.077 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.077 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.088 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:46.094 INFO:teuthology.orchestra.run.smithi162.stdout:1720 write 0x66063b3 thru 0x660df94 (0x7be2 bytes) 2024-09-16T11:13:46.101 INFO:teuthology.orchestra.run.smithi162.stdout:1721 write 0x840b0ee thru 0x8417602 (0xc515 bytes) 2024-09-16T11:13:46.107 INFO:teuthology.orchestra.run.smithi162.stdout:1722 writesame 0x8464f4f thru 0x846c6be data_size 0x27(0x7770 bytes) 2024-09-16T11:13:46.113 INFO:teuthology.orchestra.run.smithi162.stdout:1723 punch from 0xb092a2b to 0xb0946c7, (0x1c9c bytes) 2024-09-16T11:13:46.123 INFO:teuthology.orchestra.run.smithi162.stdout:1724 trunc from 0xb2cac5f to 0x64333dc 2024-09-16T11:13:46.146 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:46.137+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:46.148 INFO:teuthology.orchestra.run.smithi162.stdout:1725 punch from 0x281dca3 to 0x28286de, (0xaa3b bytes) 2024-09-16T11:13:46.148 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:46.148 INFO:teuthology.orchestra.run.smithi162.stdout:1728 writesame 0x44c4657 thru 0x44c5490 data_size 0x3(0xe3a bytes) 2024-09-16T11:13:46.151 INFO:teuthology.orchestra.run.smithi162.stdout:1729 punch from 0x1133ecb to 0x113983f, (0x5974 bytes) 2024-09-16T11:13:46.152 INFO:teuthology.orchestra.run.smithi162.stdout:1731 trunc from 0x64333dc to 0x71a0d01 2024-09-16T11:13:46.181 INFO:teuthology.orchestra.run.smithi162.stdout:1733 writesame 0xb03653a thru 0xb03f14b data_size 0x1(0x8c12 bytes) 2024-09-16T11:13:46.181 INFO:teuthology.orchestra.run.smithi162.stdout:1734 write 0x1a2a148 thru 0x1a314ca (0x7383 bytes) 2024-09-16T11:13:46.198 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.199 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.208 INFO:teuthology.orchestra.run.smithi162.stdout:1735 write 0xd9be7a5 thru 0xd9c9883 (0xb0df bytes) 2024-09-16T11:13:46.209 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:46.214 INFO:teuthology.orchestra.run.smithi162.stdout:1736 writesame 0xd25b343 thru 0xd268b99 data_size 0x1(0xd857 bytes) 2024-09-16T11:13:46.246 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:46.237+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:46.252 INFO:teuthology.orchestra.run.smithi162.stdout:1737 writesame 0x98cf6c4 thru 0x98df090 data_size 0x1(0xf9cd bytes) 2024-09-16T11:13:46.298 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:46.298 INFO:teuthology.orchestra.run.smithi162.stdout:1740 punch from 0x3837e75 to 0x3839e43, (0x1fce bytes) 2024-09-16T11:13:46.298 INFO:teuthology.orchestra.run.smithi162.stdout:1741 write 0x9add59c thru 0x9ae0a17 (0x347c bytes) 2024-09-16T11:13:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:1742 writesame 0xc43fe83 thru 0xc440a3b data_size 0x1(0xbb9 bytes) 2024-09-16T11:13:46.308 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.308 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.315 INFO:teuthology.orchestra.run.smithi162.stdout:1744 write 0x2be2fbb thru 0x2bee1c2 (0xb208 bytes) 2024-09-16T11:13:46.317 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:46.342 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:13:46.345 INFO:teuthology.orchestra.run.smithi162.stdout:1745 write 0x950a1b5 thru 0x9512e6f (0x8cbb bytes) 2024-09-16T11:13:46.347 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:46.340+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:46.357 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:46.357 INFO:teuthology.orchestra.run.smithi162.stdout:1748 write 0x933328 thru 0x939d97 (0x6a70 bytes) 2024-09-16T11:13:46.376 INFO:teuthology.orchestra.run.smithi162.stdout:1750 punch from 0xb9661c4 to 0xb96a6c7, (0x4503 bytes) 2024-09-16T11:13:46.378 INFO:teuthology.orchestra.run.smithi162.stdout:1754 trunc from 0xd9c9884 to 0xe7723ab 2024-09-16T11:13:46.388 INFO:teuthology.orchestra.run.smithi162.stdout:1756 writesame 0x1434ab8 thru 0x1437078 data_size 0x1(0x25c1 bytes) 2024-09-16T11:13:46.401 INFO:teuthology.orchestra.run.smithi162.stdout:1758 read 0x6c39cb0 thru 0x6c45271 (0xb5c2 bytes) 2024-09-16T11:13:46.401 INFO:teuthology.orchestra.run.smithi162.stdout:1759 write 0x3aec727 thru 0x3aeecc5 (0x259f bytes) 2024-09-16T11:13:46.415 INFO:teuthology.orchestra.run.smithi162.stdout:1760 write 0xe86090a thru 0xe86542e (0x4b25 bytes) 2024-09-16T11:13:46.419 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.420 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.421 INFO:teuthology.orchestra.run.smithi162.stdout:1761 write 0xe2ab260 thru 0xe2b7ff4 (0xcd95 bytes) 2024-09-16T11:13:46.427 INFO:teuthology.orchestra.run.smithi162.stdout:1762 write 0x7044de6 thru 0x704d66e (0x8889 bytes) 2024-09-16T11:13:46.430 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:46.434 INFO:teuthology.orchestra.run.smithi162.stdout:1763 read 0x1305580 thru 0x130c53e (0x6fbf bytes) 2024-09-16T11:13:46.448 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:46.441+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:46.454 INFO:teuthology.orchestra.run.smithi162.stdout:1764 trunc from 0xe86542f to 0x5e92230 2024-09-16T11:13:46.519 INFO:teuthology.orchestra.run.smithi162.stdout:1765 write 0xc50e2a8 thru 0xc51685c (0x85b5 bytes) 2024-09-16T11:13:46.531 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.531 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.540 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:46.543 INFO:teuthology.orchestra.run.smithi162.stdout:1766 trunc from 0xc51685d to 0x1cdde43 2024-09-16T11:13:46.549 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:46.541+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:46.597 INFO:teuthology.orchestra.run.smithi162.stdout:1768 write 0x48d0ee2 thru 0x48df87b (0xe99a bytes) 2024-09-16T11:13:46.601 INFO:teuthology.orchestra.run.smithi162.stdout:1769 read 0x33122df thru 0x331d3f7 (0xb119 bytes) 2024-09-16T11:13:46.602 INFO:teuthology.orchestra.run.smithi162.stdout:1770 writesame 0x1d76ef7 thru 0x1d86372 data_size 0x2(0xf47c bytes) 2024-09-16T11:13:46.622 INFO:teuthology.orchestra.run.smithi162.stdout:1771 read 0x45d63e3 thru 0x45d9e34 (0x3a52 bytes) 2024-09-16T11:13:46.622 INFO:teuthology.orchestra.run.smithi162.stdout:1772 clone 14 order 21 su 8192 sc 8 2024-09-16T11:13:46.639 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.639 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.648 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:46.649 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:46.644+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:46.751 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:46.745+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:46.795 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.795 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.804 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:46.852 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:46.845+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:46.902 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:46.902 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:46.911 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:46.953 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:46.948+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:47.054 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:47.045+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:47.057 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:47.057 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:47.066 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:47.140 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:47.164 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:47.164 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:47.173 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:47.185 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:47.180+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:47.256 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:47.248+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:47.270 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:47.270 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:47.279 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:47.285 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:47.307 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:47.329 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:47.354 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:47.356 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:47.349+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:47.373 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:47.381 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:47.381 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:47.391 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:47.393 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:47.416 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:47.438 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:47.440 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:47.440 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:47.440 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:47.440 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:47.441 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:47.441 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:47.441 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:47.441 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:47.451 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone13 from 0x48df87c (overlap 0x1cdde43) to 0x0 2024-09-16T11:13:47.457 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:47.452+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.728288730999999, 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:47.508 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:47.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:47.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.340363031000003, 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:47.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:47.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:47.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.340211701000001, 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:47.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:47.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.340160214999997, 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.527 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:47.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.340090066999998, 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:47.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:47.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.538 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.339917597000003, 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:47.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.544 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.339859275000002, 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:47.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:47.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.550 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.339707154999999, 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:47.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.556 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.339528635999997, 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:47.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:47.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:47.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.339495663000001, 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.563 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:47.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:47.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.787330171000001, 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:47.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:47.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.574 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.787307937999998, 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:47.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.787179141000003, 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:47.580 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:47.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:47.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.787120795999996, 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.586 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:47.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:47.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.787066406999998, 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.786932100000001, 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:47.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:47.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.005834579999998, 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.604 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:47.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:47.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.005776730999997, 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.610 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:47.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.615 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:47.616 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:47.616 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.005689418999999, 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:47.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:47.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:47.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:47.623 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.005451291, 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:47.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:47.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:47.629 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:47.630 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-16T11:13:47.631 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:47.631 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:47.624+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:47.731 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:47.725+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:47.742 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:47.742 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:47.752 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:47.832 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:47.824+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:47.851 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:47.851 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:47.851 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:47.851 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:47.851 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:47.861 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:47.932 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:47.925+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:47.966 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:47.966 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7881898779999998, 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.967 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:13:47.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:13:47.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.788112135, 2024-09-16T11:13:47.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:13:47.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:13:47.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7857461199999998, 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:13:47.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:13:47.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:13:47.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7857266040000002, 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.984 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7845890999999998, 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:13:47.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7845732010000002, 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.989 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:13:47.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7845489470000002, 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:47.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:47.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.997 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7845336879999998, 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:13:47.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:47.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7845183690000002, 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:48.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7824959649999998, 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7824814670000002, 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.011 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:48.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.782467134, 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.015 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:48.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:48.019 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.782453077, 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:13:48.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7824392209999997, 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.024 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:13:48.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7824253460000001, 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:13:48.028 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7824115790000001, 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:13:48.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7823640699999999, 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:13:48.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.7820782519999998, 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:48.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:13:48.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.6894144359999999, 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:13:48.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:13:48.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:13:48.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.6892652539999999, 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.052 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.058 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:48.060 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:48.053+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:48.060 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:48.061 INFO:teuthology.orchestra.run.smithi162.stdout:1773 trunc from 0x48df87c to 0xd69d1e6 2024-09-16T11:13:48.061 INFO:teuthology.orchestra.run.smithi162.stdout:1774 writesame 0xc90f7a2 thru 0xc91e0d5 data_size 0x96(0xe934 bytes) 2024-09-16T11:13:48.062 INFO:teuthology.orchestra.run.smithi162.stdout:1775 write 0x8058dee thru 0x805bd5f (0x2f72 bytes) 2024-09-16T11:13:48.062 INFO:teuthology.orchestra.run.smithi162.stdout:1777 read 0x5004e2a thru 0x501306e (0xe245 bytes) 2024-09-16T11:13:48.062 INFO:teuthology.orchestra.run.smithi162.stdout:1779 read 0x27b3ca4 thru 0x27bb067 (0x73c4 bytes) 2024-09-16T11:13:48.062 INFO:teuthology.orchestra.run.smithi162.stdout:1781 read 0x9630255 thru 0x963f13c (0xeee8 bytes) 2024-09-16T11:13:48.062 INFO:teuthology.orchestra.run.smithi162.stdout:1782 trunc from 0xd69d1e6 to 0x9d64430 2024-09-16T11:13:48.079 INFO:teuthology.orchestra.run.smithi162.stdout:1783 read 0x7c670c3 thru 0x7c70e6d (0x9dab bytes) 2024-09-16T11:13:48.079 INFO:teuthology.orchestra.run.smithi162.stdout:1785 punch from 0x40ea766 to 0x40f5986, (0xb220 bytes) 2024-09-16T11:13:48.079 INFO:teuthology.orchestra.run.smithi162.stdout:1787 read 0x5e0b9b0 thru 0x5e160d0 (0xa721 bytes) 2024-09-16T11:13:48.095 INFO:teuthology.orchestra.run.smithi162.stdout:1788 trunc from 0x9d64430 to 0xb35f7b 2024-09-16T11:13:48.160 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:48.153+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49903 4.12 4.272502d2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156391+0000", 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039144501999999998, 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.03917238, 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49903 2024-09-16T11:13:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156391+0000", 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156391+0000", 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156391+0000", 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8500000000000001e-07 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156394+0000", 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156402+0000", 2024-09-16T11:13:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8159999999999994e-06 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156406+0000", 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0160000000000002e-06 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49989 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157016+0000", 2024-09-16T11:13:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038518838999999999, 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.038587192999999999, 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49989 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157016+0000", 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157016+0000", 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157017+0000", 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3499999999999996e-07 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157023+0000", 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7540000000000003e-06 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157029+0000", 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4080000000000003e-06 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157032+0000", 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2590000000000001e-06 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50250 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159568+0000", 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035967398999999997, 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036069840999999998, 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50250 2024-09-16T11:13:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159568+0000", 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159568+0000", 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159568+0000", 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159570+0000", 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9599999999999999e-06 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159576+0000", 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9159999999999998e-06 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159579+0000", 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3019999999999999e-06 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50480 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170219+0000", 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025316704999999998, 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025472873, 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50480 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170219+0000", 2024-09-16T11:13:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170219+0000", 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170219+0000", 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170221+0000", 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2390000000000001e-06 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170227+0000", 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1e-06 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170231+0000", 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8e-06 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49907 4.12 4.272502d2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156409+0000", 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039125774000000002, 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039336309999999999, 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49907 2024-09-16T11:13:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156409+0000", 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156409+0000", 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156410+0000", 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.84e-07 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156412+0000", 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9530000000000002e-06 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156418+0000", 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9540000000000001e-06 2024-09-16T11:13:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156421+0000", 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4240000000000002e-06 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49990 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157036+0000", 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038498984, 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.038765663999999998, 2024-09-16T11:13:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49990 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157036+0000", 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157036+0000", 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157037+0000", 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4499999999999997e-07 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157039+0000", 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9199999999999998e-06 2024-09-16T11:13:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157045+0000", 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7810000000000002e-06 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157048+0000", 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.084e-06 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50251 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159584+0000", 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035951556000000003, 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036272184999999998, 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50251 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159584+0000", 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159584+0000", 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159584+0000", 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159586+0000", 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2699999999999999e-06 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159596+0000", 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2029999999999998e-06 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159625+0000", 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9016000000000001e-05 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50481 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170235+0000", 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025299822999999999, 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025668013999999999, 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50481 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170235+0000", 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170235+0000", 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170236+0000", 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170238+0000", 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.182e-06 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170244+0000", 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6559999999999997e-06 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170247+0000", 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5889999999999999e-06 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49935 4.12 4.272502d2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156425+0000", 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039110165000000002, 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039530191999999999, 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49935 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156425+0000", 2024-09-16T11:13:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156425+0000", 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156425+0000", 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156427+0000", 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6339999999999999e-06 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156433+0000", 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9610000000000002e-06 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156436+0000", 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1990000000000002e-06 2024-09-16T11:13:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49991 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157051+0000", 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038483884000000003, 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.038947582000000001, 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49991 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157051+0000", 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157051+0000", 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157052+0000", 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2399999999999998e-07 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157054+0000", 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6759999999999999e-06 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157059+0000", 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6300000000000003e-06 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157062+0000", 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1549999999999999e-06 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50252 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159631+0000", 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035904179000000001, 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036421528000000002, 2024-09-16T11:13:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50252 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159631+0000", 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159631+0000", 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159632+0000", 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159634+0000", 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.751e-06 2024-09-16T11:13:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159641+0000", 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7290000000000003e-06 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159645+0000", 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0289999999999999e-06 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50482 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170253+0000", 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025282671999999999, 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025853538999999998, 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50482 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170253+0000", 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170253+0000", 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170253+0000", 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4799999999999999e-07 2024-09-16T11:13:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170255+0000", 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.322e-06 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170261+0000", 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170265+0000", 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49992 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157066+0000", 2024-09-16T11:13:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038468950000000002, 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039095284000000001, 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49992 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157066+0000", 2024-09-16T11:13:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157066+0000", 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157067+0000", 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6499999999999999e-07 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157069+0000", 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157075+0000", 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8799999999999996e-06 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157078+0000", 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1389999999999999e-06 2024-09-16T11:13:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50253 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159649+0000", 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035886288000000002, 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036565831, 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50253 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159649+0000", 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159649+0000", 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159649+0000", 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159651+0000", 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159657+0000", 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6710000000000004e-06 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159660+0000", 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50483 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170269+0000", 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025266143000000001, 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.025979711999999999, 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50483 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170269+0000", 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170269+0000", 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170270+0000", 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8700000000000005e-07 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170272+0000", 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4420000000000001e-06 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170278+0000", 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6420000000000004e-06 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170281+0000", 2024-09-16T11:13:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5930000000000002e-06 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49993 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157081+0000", 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038454521999999998, 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039228460999999999, 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49993 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157081+0000", 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157081+0000", 2024-09-16T11:13:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157081+0000", 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3099999999999998e-07 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157082+0000", 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1620000000000001e-06 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157085+0000", 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0060000000000001e-06 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157087+0000", 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8700000000000001e-06 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50254 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159664+0000", 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035871556999999998, 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036699428999999999, 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.265 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50254 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159664+0000", 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159664+0000", 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159664+0000", 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159666+0000", 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.024e-06 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159671+0000", 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159675+0000", 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7730000000000001e-06 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50484 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170286+0000", 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025249572000000001, 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026132741000000001, 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50484 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170286+0000", 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170286+0000", 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170286+0000", 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.01e-07 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170288+0000", 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170294+0000", 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8629999999999996e-06 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170297+0000", 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5140000000000002e-06 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49994 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157089+0000", 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038446173, 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039387973, 2024-09-16T11:13:48.272 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49994 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157089+0000", 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157089+0000", 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157089+0000", 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157091+0000", 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.052e-06 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157093+0000", 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9019999999999999e-06 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157095+0000", 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8679999999999999e-06 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50255 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:48.275 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035839098999999999, 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036834901000000003, 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50255 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159697+0000", 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2e-07 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159699+0000", 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.114e-06 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159705+0000", 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9970000000000004e-06 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159709+0000", 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.278 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50485 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170302+0000", 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025233604, 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026282598000000001, 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50485 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170302+0000", 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170302+0000", 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170302+0000", 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170304+0000", 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3870000000000002e-06 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170310+0000", 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170312+0000", 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3269999999999999e-06 2024-09-16T11:13:48.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49995 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157097+0000", 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038438057999999997, 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039539572000000002, 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.282 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49995 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157097+0000", 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157097+0000", 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157098+0000", 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157099+0000", 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.006e-06 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157102+0000", 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0130000000000002e-06 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157103+0000", 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8050000000000001e-06 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50211 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157553+0000", 2024-09-16T11:13:48.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.037982344000000001, 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039127081000000001, 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50211 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157553+0000", 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157553+0000", 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157553+0000", 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157555+0000", 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.296e-06 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157558+0000", 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8190000000000002e-06 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157560+0000", 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035822514999999999, 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.036994722000000001, 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:13:48.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:13:48.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50486 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170314+0000", 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025220985000000001, 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026441868, 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50486 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170314+0000", 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170314+0000", 2024-09-16T11:13:48.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170315+0000", 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7300000000000002e-07 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170316+0000", 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.95e-06 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170323+0000", 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4489999999999996e-06 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170328+0000", 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5079999999999996e-06 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49996 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.295 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157105+0000", 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038430091999999999, 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039705022, 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49996 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157105+0000", 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157105+0000", 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157106+0000", 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157106+0000", 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6899999999999996e-07 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157109+0000", 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9519999999999999e-06 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157111+0000", 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.807e-06 2024-09-16T11:13:48.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035787461, 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037117810000000001, 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50487 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170332+0000", 2024-09-16T11:13:48.302 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025202796, 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.02658462, 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50487 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170332+0000", 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170332+0000", 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170333+0000", 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170335+0000", 2024-09-16T11:13:48.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4619999999999999e-06 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170345+0000", 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5419999999999994e-06 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170349+0000", 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7249999999999999e-06 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49997 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157113+0000", 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038422283000000002, 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039857434999999997, 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49997 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157113+0000", 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157113+0000", 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157113+0000", 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8500000000000002e-07 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157115+0000", 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.457e-06 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157118+0000", 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9869999999999999e-06 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157120+0000", 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1940000000000001e-06 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035771026999999997, 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037257169, 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.309 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:13:48.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:13:48.311 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50488 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170353+0000", 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025182435999999999, 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026717560000000001, 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50488 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170353+0000", 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170353+0000", 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170353+0000", 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170356+0000", 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8660000000000002e-06 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170364+0000", 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170374+0000", 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9210000000000004e-06 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49998 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157124+0000", 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038411643000000002, 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039984533000000003, 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49998 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157124+0000", 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157124+0000", 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157124+0000", 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157125+0000", 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157128+0000", 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0819999999999999e-06 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157130+0000", 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0150000000000002e-06 2024-09-16T11:13:48.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035754718999999997, 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037378770999999998, 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.319 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:13:48.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50489 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170507+0000", 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025027796000000001, 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026696905, 2024-09-16T11:13:48.322 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50489 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170507+0000", 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170507+0000", 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170508+0000", 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3500000000000005e-07 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170513+0000", 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7350000000000004e-06 2024-09-16T11:13:48.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170524+0000", 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.119e-05 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170530+0000", 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9200000000000001e-06 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49999 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157132+0000", 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038403087000000002, 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040122326999999999, 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49999 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157132+0000", 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157132+0000", 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157133+0000", 2024-09-16T11:13:48.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157134+0000", 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.274e-06 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157137+0000", 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9699999999999999e-06 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157139+0000", 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.007e-06 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50217 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157604+0000", 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.037931342999999999, 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039746034999999999, 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.329 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50217 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157604+0000", 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157604+0000", 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157604+0000", 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157606+0000", 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0719999999999998e-06 2024-09-16T11:13:48.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157611+0000", 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.113e-06 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157615+0000", 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035739153000000003, 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037604076, 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:13:48.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50490 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170536+0000", 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024999433000000001, 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026925896000000001, 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50490 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170536+0000", 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170536+0000", 2024-09-16T11:13:48.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170536+0000", 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170557+0000", 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.039e-05 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170565+0000", 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2369999999999992e-06 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170570+0000", 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8940000000000001e-06 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50000 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157141+0000", 2024-09-16T11:13:48.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038394623000000003, 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040375309999999998, 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50000 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157141+0000", 2024-09-16T11:13:48.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157141+0000", 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157141+0000", 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157142+0000", 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.285e-06 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157147+0000", 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5210000000000004e-06 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157150+0000", 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1149999999999998e-06 2024-09-16T11:13:48.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035004659, 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037042009000000001, 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50491 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170632+0000", 2024-09-16T11:13:48.346 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024903670999999999, 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.026996464000000001, 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50491 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170632+0000", 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170632+0000", 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170632+0000", 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2399999999999997e-07 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170636+0000", 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.72e-06 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170646+0000", 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8069999999999994e-06 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170651+0000", 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2569999999999998e-06 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50001 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157154+0000", 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038381580999999998, 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040540936, 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.350 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50001 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157154+0000", 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157154+0000", 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157154+0000", 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157160+0000", 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8200000000000002e-06 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157167+0000", 2024-09-16T11:13:48.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157171+0000", 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7339999999999999e-06 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.353 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.034988630999999999, 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037203152000000003, 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:13:48.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50492 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170657+0000", 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024878201999999999, 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027144561000000001, 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50492 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.357 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170657+0000", 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170657+0000", 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170658+0000", 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6400000000000002e-07 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170660+0000", 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4140000000000001e-06 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170666+0000", 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3339999999999998e-06 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170671+0000", 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2239999999999997e-06 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50002 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157189+0000", 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038346572000000002, 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040654726000000002, 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.360 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50002 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157189+0000", 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157189+0000", 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157189+0000", 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3399999999999999e-07 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157192+0000", 2024-09-16T11:13:48.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2800000000000002e-06 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157198+0000", 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157202+0000", 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.895e-06 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50222 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157915+0000", 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.037619910999999999, 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039972375999999997, 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50222 2024-09-16T11:13:48.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157915+0000", 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157915+0000", 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157916+0000", 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7400000000000002e-07 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157920+0000", 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.895e-06 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157931+0000", 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0852e-05 2024-09-16T11:13:48.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157936+0000", 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1270000000000002e-06 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.034978800999999997, 2024-09-16T11:13:48.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.037388139000000001, 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:13:48.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50493 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170674+0000", 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024860872999999999, 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027327147999999999, 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50493 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170674+0000", 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170674+0000", 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170675+0000", 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170677+0000", 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170684+0000", 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8730000000000001e-06 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170688+0000", 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8419999999999998e-06 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49742 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 958464~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156141+0000", 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039393810000000001, 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041926116999999999, 2024-09-16T11:13:48.374 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49742 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156141+0000", 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156141+0000", 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156142+0000", 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.48e-07 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156145+0000", 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5090000000000002e-06 2024-09-16T11:13:48.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156148+0000", 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156150+0000", 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.796e-06 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197754+0000", 2024-09-16T11:13:48.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041603569999999999 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197806+0000", 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2163000000000002e-05 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50003 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.378 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157206+0000", 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038329462000000002, 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040928121999999997, 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50003 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157206+0000", 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157206+0000", 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157206+0000", 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4799999999999998e-07 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157209+0000", 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1639999999999999e-06 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157214+0000", 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.818e-06 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157218+0000", 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6169999999999998e-06 2024-09-16T11:13:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031668605000000002, 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034327027000000003, 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.382 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:13:48.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50494 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170692+0000", 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024843446000000002, 2024-09-16T11:13:48.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027552027, 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50494 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170692+0000", 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170692+0000", 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170692+0000", 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170694+0000", 2024-09-16T11:13:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.12e-06 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170701+0000", 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8900000000000001e-06 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170705+0000", 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5300000000000001e-06 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49743 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156152+0000", 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039383168000000003, 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042131351999999997, 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49743 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156152+0000", 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156152+0000", 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156152+0000", 2024-09-16T11:13:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3799999999999999e-07 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156153+0000", 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.111e-06 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156156+0000", 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0089999999999999e-06 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156158+0000", 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7519999999999999e-06 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50004 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157222+0000", 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038313532999999997, 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041091615999999997, 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50004 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157222+0000", 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157222+0000", 2024-09-16T11:13:48.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157222+0000", 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5899999999999996e-07 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157224+0000", 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.809e-06 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157230+0000", 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157234+0000", 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7680000000000002e-06 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.395 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031645254999999997, 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034457134, 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:13:48.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50495 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170709+0000", 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024825854000000001, 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027667092000000001, 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50495 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170709+0000", 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170709+0000", 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170710+0000", 2024-09-16T11:13:48.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6199999999999999e-07 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170712+0000", 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.801e-06 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170718+0000", 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9560000000000002e-06 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170721+0000", 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8569999999999997e-06 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49744 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156160+0000", 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039375151999999997, 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042250402999999999, 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.402 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49744 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156160+0000", 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156160+0000", 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156160+0000", 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3200000000000001e-07 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156161+0000", 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.119e-06 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156164+0000", 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9500000000000001e-06 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156166+0000", 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7150000000000001e-06 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50005 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157238+0000", 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038297539999999998, 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041224930999999999, 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50005 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157238+0000", 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157238+0000", 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157238+0000", 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2200000000000004e-07 2024-09-16T11:13:48.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157240+0000", 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2589999999999999e-06 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157247+0000", 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2729999999999999e-06 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157250+0000", 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.67e-06 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031625954999999997, 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034605314999999998, 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:13:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50496 4.1c 4.811e051c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170726+0000", 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.024808909000000001, 2024-09-16T11:13:48.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.027848650999999999, 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50496 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170726+0000", 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170726+0000", 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170727+0000", 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170729+0000", 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.001e-06 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170734+0000", 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3720000000000004e-06 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170738+0000", 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7730000000000001e-06 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.415 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49745 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156168+0000", 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039367391000000002, 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042448452999999997, 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49745 2024-09-16T11:13:48.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156168+0000", 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156168+0000", 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156168+0000", 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156169+0000", 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6899999999999996e-07 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156172+0000", 2024-09-16T11:13:48.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.802e-06 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156174+0000", 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50006 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157254+0000", 2024-09-16T11:13:48.419 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038281074999999998, 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041391952000000003, 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50006 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157254+0000", 2024-09-16T11:13:48.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157254+0000", 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157255+0000", 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157257+0000", 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.968e-06 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157264+0000", 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157268+0000", 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031608472999999998, 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034746154000000001, 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.423 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49746 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156175+0000", 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039359841999999999, 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042537927000000003, 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.426 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49746 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156175+0000", 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156175+0000", 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156176+0000", 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.48e-07 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156177+0000", 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0589999999999999e-06 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156180+0000", 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8260000000000001e-06 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156185+0000", 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6080000000000003e-06 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.429 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50007 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157272+0000", 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038263580999999998, 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041491673999999999, 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50007 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157272+0000", 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157272+0000", 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157272+0000", 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9699999999999995e-07 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157274+0000", 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0109999999999999e-06 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157280+0000", 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157284+0000", 2024-09-16T11:13:48.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6349999999999999e-06 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50229 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159188+0000", 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036347369999999997, 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.039633902999999998, 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50229 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159188+0000", 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159188+0000", 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159188+0000", 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.01e-07 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159190+0000", 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0310000000000001e-06 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159196+0000", 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7389999999999996e-06 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159199+0000", 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0970000000000002e-06 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4.a0a9f752 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:48.436 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031577894000000002, 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034901809999999998, 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:48.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.439 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49747 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156187+0000", 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039347981999999997, 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042703385000000003, 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49747 2024-09-16T11:13:48.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156187+0000", 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156187+0000", 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156188+0000", 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156189+0000", 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1170000000000001e-06 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156192+0000", 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3739999999999998e-06 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156199+0000", 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5590000000000002e-06 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50008 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157288+0000", 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038247681999999998, 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041635336000000002, 2024-09-16T11:13:48.443 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50008 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157288+0000", 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157288+0000", 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157288+0000", 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157290+0000", 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157295+0000", 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4240000000000001e-06 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157299+0000", 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3459999999999998e-06 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4.a0a9f752 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031456076999999999, 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034938125, 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:13:48.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49748 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156201+0000", 2024-09-16T11:13:48.450 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039334514000000001, 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042844307999999998, 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49748 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156201+0000", 2024-09-16T11:13:48.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156201+0000", 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156201+0000", 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9299999999999999e-07 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156202+0000", 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.401e-06 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156206+0000", 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1140000000000002e-06 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156207+0000", 2024-09-16T11:13:48.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50009 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157303+0000", 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038232615999999997, 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041770863999999998, 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.454 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50009 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157303+0000", 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157303+0000", 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157303+0000", 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3399999999999999e-07 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157305+0000", 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9470000000000002e-06 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157311+0000", 2024-09-16T11:13:48.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6729999999999997e-06 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157314+0000", 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4.a0a9f752 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:48.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.031014907000000001, 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034582473000000002, 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:48.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49749 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156209+0000", 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039325964999999997, 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042943925000000001, 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49749 2024-09-16T11:13:48.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156209+0000", 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156209+0000", 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156210+0000", 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.35e-07 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156211+0000", 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0559999999999999e-06 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156213+0000", 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9069999999999999e-06 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156215+0000", 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50010 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157318+0000", 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038216808999999997, 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041888498000000003, 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.464 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50010 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157318+0000", 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157318+0000", 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157319+0000", 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6300000000000005e-07 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157321+0000", 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0899999999999999e-06 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157327+0000", 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5380000000000002e-06 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157330+0000", 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.030384035, 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034107867, 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:13:48.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49750 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156217+0000", 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039318100000000002, 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043083551999999997, 2024-09-16T11:13:48.471 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49750 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156217+0000", 2024-09-16T11:13:48.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156217+0000", 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156217+0000", 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4999999999999999e-07 2024-09-16T11:13:48.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156218+0000", 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.64e-07 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156221+0000", 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8760000000000001e-06 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156223+0000", 2024-09-16T11:13:48.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.671e-06 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50011 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157335+0000", 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038199812, 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042011626000000003, 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50011 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157335+0000", 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157335+0000", 2024-09-16T11:13:48.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157336+0000", 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3399999999999999e-07 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157338+0000", 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0049999999999999e-06 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157348+0000", 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0196e-05 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157352+0000", 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7790000000000002e-06 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.478 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50240 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159271+0000", 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036264519000000002, 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040116476999999998, 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50240 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159271+0000", 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159271+0000", 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159271+0000", 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:13:48.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159273+0000", 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0609999999999998e-06 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159279+0000", 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9440000000000002e-06 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159283+0000", 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3689999999999998e-06 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.030355752999999999, 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034241709000000002, 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.485 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49751 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156225+0000", 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.03931055, 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043252835000000003, 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49751 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156225+0000", 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156225+0000", 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156225+0000", 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156226+0000", 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2800000000000005e-07 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156229+0000", 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.993e-06 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156231+0000", 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9810000000000002e-06 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50186 4.12 4.a0a9f752 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157357+0000", 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038178304000000003, 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042174705, 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50186 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157357+0000", 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157357+0000", 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157357+0000", 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3200000000000005e-07 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157359+0000", 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.843e-06 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157367+0000", 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1940000000000001e-06 2024-09-16T11:13:48.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157370+0000", 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3610000000000001e-06 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50241 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159287+0000", 2024-09-16T11:13:48.492 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036248013000000003, 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040303888000000003, 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50241 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159287+0000", 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159287+0000", 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159288+0000", 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159289+0000", 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8959999999999999e-06 2024-09-16T11:13:48.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159297+0000", 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3529999999999997e-06 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159303+0000", 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6640000000000003e-06 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.030340525, 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034449352000000003, 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:13:48.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49752 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156234+0000", 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039301271999999998, 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043464438000000001, 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49752 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156234+0000", 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156234+0000", 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156234+0000", 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156241+0000", 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5899999999999996e-06 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156246+0000", 2024-09-16T11:13:48.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7949999999999998e-06 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156248+0000", 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2060000000000001e-06 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50242 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159306+0000", 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036228802999999997, 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040448451000000003, 2024-09-16T11:13:48.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50242 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159306+0000", 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159306+0000", 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159307+0000", 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3500000000000002e-07 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159309+0000", 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159316+0000", 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2640000000000002e-06 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159319+0000", 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4620000000000001e-06 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4.272502d2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.505 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.030327623000000001, 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.034602941999999998, 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:13:48.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49753 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156250+0000", 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039284982000000003, 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043619877000000001, 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49753 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156250+0000", 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156250+0000", 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156251+0000", 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0800000000000001e-07 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156252+0000", 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.226e-06 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156255+0000", 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0280000000000001e-06 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156257+0000", 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:13:48.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50243 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159323+0000", 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036212209000000002, 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040602841000000001, 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50243 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159323+0000", 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159323+0000", 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159323+0000", 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9400000000000001e-07 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159325+0000", 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9570000000000001e-06 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159331+0000", 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9479999999999996e-06 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159344+0000", 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.292e-05 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49754 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.515 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156258+0000", 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039276761, 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043726538000000002, 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49754 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156258+0000", 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156258+0000", 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156259+0000", 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156262+0000", 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.507e-06 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156265+0000", 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1719999999999999e-06 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156267+0000", 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9460000000000001e-06 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.518 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50192 4.12 4.a0a9f752 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157411+0000", 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038123935999999997, 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.042629289000000001, 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50192 2024-09-16T11:13:48.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157411+0000", 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157411+0000", 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157412+0000", 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.89e-07 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157414+0000", 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.345e-06 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157420+0000", 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9769999999999997e-06 2024-09-16T11:13:48.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157423+0000", 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2669999999999998e-06 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50244 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159349+0000", 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036186206999999998, 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040747877000000002, 2024-09-16T11:13:48.522 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50244 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159349+0000", 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159349+0000", 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159349+0000", 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159356+0000", 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1970000000000001e-06 2024-09-16T11:13:48.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159470+0000", 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114096 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159477+0000", 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.447e-06 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49755 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156269+0000", 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039265794, 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043882283000000001, 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49755 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156269+0000", 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156269+0000", 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156270+0000", 2024-09-16T11:13:48.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156271+0000", 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1009999999999999e-06 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156274+0000", 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156302+0000", 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7684000000000001e-05 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49984 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156922+0000", 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038613567000000001, 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043280948999999999, 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49984 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156922+0000", 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156922+0000", 2024-09-16T11:13:48.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156922+0000", 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156924+0000", 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1100000000000001e-06 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156930+0000", 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8510000000000004e-06 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156933+0000", 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.354e-06 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50245 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159484+0000", 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036050879000000001, 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040761614000000002, 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50245 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159484+0000", 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159484+0000", 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159485+0000", 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7400000000000002e-07 2024-09-16T11:13:48.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159489+0000", 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4379999999999999e-06 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159497+0000", 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9419999999999995e-06 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159500+0000", 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9199999999999997e-06 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49869 4.12 4.a0a9f752 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156307+0000", 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039228332999999997, 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044002148999999997, 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49869 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156307+0000", 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156307+0000", 2024-09-16T11:13:48.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156307+0000", 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4499999999999997e-07 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156310+0000", 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7180000000000001e-06 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156317+0000", 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5429999999999999e-06 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156320+0000", 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6109999999999998e-06 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49985 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156937+0000", 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038598082999999998, 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043427427999999997, 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49985 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156937+0000", 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156937+0000", 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156938+0000", 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.46e-07 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156949+0000", 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.188e-05 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156956+0000", 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6390000000000003e-06 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156960+0000", 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7409999999999998e-06 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50246 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159505+0000", 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036029950999999998, 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.040916307999999998, 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50246 2024-09-16T11:13:48.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159505+0000", 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159505+0000", 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159506+0000", 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7399999999999998e-07 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159508+0000", 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159514+0000", 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4910000000000002e-06 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159518+0000", 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8619999999999996e-06 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49876 4.12 4.a0a9f752 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156325+0000", 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039210056, 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044150596, 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49876 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156325+0000", 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156325+0000", 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156326+0000", 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1900000000000002e-07 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156328+0000", 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156334+0000", 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6400000000000002e-06 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156337+0000", 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4369999999999999e-06 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49986 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.549 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156964+0000", 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038570938999999999, 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043562478000000002, 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49986 2024-09-16T11:13:48.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156964+0000", 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156964+0000", 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156965+0000", 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8599999999999998e-07 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156967+0000", 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0650000000000001e-06 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156973+0000", 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7710000000000003e-06 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156976+0000", 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.287e-06 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50196 4.12 4.a0a9f752 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.157458+0000", 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038077471000000002, 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043125577999999998, 2024-09-16T11:13:48.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50196 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157458+0000", 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157458+0000", 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157458+0000", 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157459+0000", 2024-09-16T11:13:48.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3069999999999999e-06 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157462+0000", 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9249999999999999e-06 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157464+0000", 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0779999999999998e-06 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50247 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159522+0000", 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.036013041000000003, 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041113032000000001, 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50247 2024-09-16T11:13:48.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159522+0000", 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159522+0000", 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159523+0000", 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.53e-07 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159525+0000", 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159530+0000", 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7910000000000001e-06 2024-09-16T11:13:48.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159534+0000", 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6339999999999999e-06 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50477 4.1c 4:38a07881:::rbd_data.11b5eaebe1b2.0000000000000009:11 [read 319488~8192 out=8192b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.169046+0000", 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.026488985999999999, 2024-09-16T11:13:48.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.031556927999999998, 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50477 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.169046+0000", 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.169046+0000", 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.169047+0000", 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0759999999999999e-06 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.169052+0000", 2024-09-16T11:13:48.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9579999999999998e-06 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.169070+0000", 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8107000000000001e-05 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.169077+0000", 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6560000000000003e-06 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197959+0000", 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.028881713999999999 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198033+0000", 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4042999999999996e-05 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.200603+0000", 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025703739999999998 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49880 4.12 4.a0a9f752 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156341+0000", 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039194689999999997, 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044436047999999999, 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49880 2024-09-16T11:13:48.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156341+0000", 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156341+0000", 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156341+0000", 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6800000000000001e-07 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156343+0000", 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156348+0000", 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6489999999999996e-06 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156352+0000", 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4020000000000002e-06 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49987 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156980+0000", 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038555514999999999, 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.043854983, 2024-09-16T11:13:48.568 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49987 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156980+0000", 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156980+0000", 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156980+0000", 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156982+0000", 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:13:48.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156988+0000", 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9349999999999999e-06 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156991+0000", 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9979999999999999e-06 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50248 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159538+0000", 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035997622999999999, 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041345113000000003, 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50248 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159538+0000", 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159538+0000", 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159538+0000", 2024-09-16T11:13:48.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159540+0000", 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.165e-06 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159546+0000", 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3890000000000004e-06 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159549+0000", 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6799999999999999e-06 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50478 4.1c 4:38a07881:::rbd_data.11b5eaebe1b2.0000000000000009:11 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170167+0000", 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025368552999999999, 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030778778999999999, 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50478 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170167+0000", 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170167+0000", 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170168+0000", 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.850000000000001e-07 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170173+0000", 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1630000000000004e-06 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170185+0000", 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1815e-05 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170191+0000", 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.200623+0000", 2024-09-16T11:13:48.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030432612000000001 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.200716+0000", 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2548999999999998e-05 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49897 4.12 4.272502d2 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156355+0000", 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.039180025, 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044632480000000002, 2024-09-16T11:13:48.579 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49897 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156355+0000", 2024-09-16T11:13:48.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156355+0000", 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156356+0000", 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6899999999999998e-07 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156358+0000", 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8759999999999999e-06 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156363+0000", 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5280000000000003e-06 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156379+0000", 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6334e-05 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49988 4.12 4.19920b12 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.156995+0000", 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.038540653000000001, 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044048096000000002, 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49988 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156995+0000", 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156995+0000", 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156995+0000", 2024-09-16T11:13:48.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6100000000000001e-07 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.156997+0000", 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6300000000000001e-06 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157002+0000", 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7930000000000003e-06 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.157012+0000", 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1300000000000007e-06 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50249 4.12 4.19920b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159553+0000", 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.035982179000000003, 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.041537045000000002, 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:48.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50249 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159553+0000", 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159553+0000", 2024-09-16T11:13:48.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159553+0000", 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159555+0000", 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.888e-06 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159561+0000", 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4110000000000004e-06 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159564+0000", 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.371e-06 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50479 4.1c 4:38a07881:::rbd_data.11b5eaebe1b2.0000000000000009:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.589 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.170200+0000", 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.025335600999999999, 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030938680999999999, 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50479 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170200+0000", 2024-09-16T11:13:48.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170200+0000", 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170200+0000", 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7899999999999998e-07 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170203+0000", 2024-09-16T11:13:48.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3530000000000002e-06 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170209+0000", 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.170214+0000", 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4240000000000003e-06 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.201099+0000", 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030885505000000001 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 111 2024-09-16T11:13:48.593 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:48.594 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:13:48.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:48.597 INFO:teuthology.orchestra.run.smithi162.stdout:1791 read 0x52a66a thru 0x534b4b (0xa4e2 bytes) 2024-09-16T11:13:48.597 INFO:teuthology.orchestra.run.smithi162.stdout:1793 write 0x355974b thru 0x3565d36 (0xc5ec bytes) 2024-09-16T11:13:48.597 INFO:teuthology.orchestra.run.smithi162.stdout:1794 read 0x24d0ff8 thru 0x24dc57b (0xb584 bytes) 2024-09-16T11:13:48.597 INFO:teuthology.orchestra.run.smithi162.stdout:1795 writesame 0x3b8e0ce thru 0x3b909b5 data_size 0x2c(0x28e8 bytes) 2024-09-16T11:13:48.597 INFO:teuthology.orchestra.run.smithi162.stdout:1796 punch from 0xf8df07 to 0xf96d48, (0x8e41 bytes) 2024-09-16T11:13:48.597 INFO:teuthology.orchestra.run.smithi162.stdout:1797 writesame 0x846f249 thru 0x8477660 data_size 0xc(0x8418 bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1799 writesame 0x50e2d71 thru 0x50e9b8a data_size 0x11(0x6e1a bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1800 punch from 0x1080c37 to 0x108aad0, (0x9e99 bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1801 write 0x38b9b04 thru 0x38bd2ae (0x37ab bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1802 write 0x849df00 thru 0x84acf02 (0xf003 bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1803 read 0x4206a31 thru 0x420926c (0x283c bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1805 writesame 0xa37bc57 thru 0xa383d2f data_size 0xf(0x80d9 bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1807 write 0x2f93591 thru 0x2f9f32b (0xbd9b bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1809 trunc from 0xa383d30 to 0x74f6802 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1812 read 0x3eeca42 thru 0x3ef3faf (0x756e bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1814 writesame 0x6f105d0 thru 0x6f19130 data_size 0x1(0x8b61 bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1815 read 0x6e87c8c thru 0x6e9341d (0xb792 bytes) 2024-09-16T11:13:48.598 INFO:teuthology.orchestra.run.smithi162.stdout:1816 trunc from 0x74f6802 to 0xc649083 2024-09-16T11:13:48.599 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:48.588+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:48.604 INFO:teuthology.orchestra.run.smithi162.stdout:1817 write 0x1c4fa8a thru 0x1c5ede8 (0xf35f bytes) 2024-09-16T11:13:48.626 INFO:teuthology.orchestra.run.smithi162.stdout:1820 read 0x3b9e0f1 thru 0x3b9ee5b (0xd6b bytes) 2024-09-16T11:13:48.626 INFO:teuthology.orchestra.run.smithi162.stdout:1821 read 0xc0264da thru 0xc02af17 (0x4a3e bytes) 2024-09-16T11:13:48.636 INFO:teuthology.orchestra.run.smithi162.stdout:1822 trunc from 0xc649083 to 0x7380f0b 2024-09-16T11:13:48.648 INFO:teuthology.orchestra.run.smithi162.stdout:1823 punch from 0x54363f to 0x546cda, (0x369b bytes) 2024-09-16T11:13:48.671 INFO:teuthology.orchestra.run.smithi162.stdout:1825 write 0xda210c9 thru 0xda2b0d0 (0xa008 bytes) 2024-09-16T11:13:48.689 INFO:teuthology.orchestra.run.smithi162.stdout:1826 read 0x17d52c thru 0x17dfe8 (0xabd bytes) 2024-09-16T11:13:48.690 INFO:teuthology.orchestra.run.smithi162.stdout:1828 punch from 0x3d65468 to 0x3d6fed3, (0xaa6b bytes) 2024-09-16T11:13:48.690 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:48.690 INFO:teuthology.orchestra.run.smithi162.stdout:1830 write 0x31b30c8 thru 0x31bef39 (0xbe72 bytes) 2024-09-16T11:13:48.695 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:48.689+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:48.695 INFO:teuthology.orchestra.run.smithi162.stdout:1831 write 0x15f5296 thru 0x15f9b25 (0x4890 bytes) 2024-09-16T11:13:48.698 INFO:teuthology.orchestra.run.smithi162.stdout:1833 writesame 0xb49ae8c thru 0xb49f50c data_size 0x1(0x4681 bytes) 2024-09-16T11:13:48.713 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:48.714 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:48.719 INFO:teuthology.orchestra.run.smithi162.stdout:1834 trunc from 0xda2b0d1 to 0x7b3c943 2024-09-16T11:13:48.727 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:48.746 INFO:teuthology.orchestra.run.smithi162.stdout:1836 trunc from 0x7b3c943 to 0x2115a34 2024-09-16T11:13:48.770 INFO:teuthology.orchestra.run.smithi162.stdout:1837 write 0xcf0abe thru 0xcfa30e (0x9851 bytes) 2024-09-16T11:13:48.787 INFO:teuthology.orchestra.run.smithi162.stdout:1838 trunc from 0x2115a34 to 0xaedc0fc 2024-09-16T11:13:48.796 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:48.789+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:48.796 INFO:teuthology.orchestra.run.smithi162.stdout:1840 writesame 0x925ce3c thru 0x925dbbd data_size 0x1a(0xd82 bytes) 2024-09-16T11:13:48.802 INFO:teuthology.orchestra.run.smithi162.stdout:1841 read 0x63d3f78 thru 0x63dfa7b (0xbb04 bytes) 2024-09-16T11:13:48.802 INFO:teuthology.orchestra.run.smithi162.stdout:1842 punch from 0x9ef1f76 to 0x9efd1c0, (0xb24a bytes) 2024-09-16T11:13:48.802 INFO:teuthology.orchestra.run.smithi162.stdout:1843 writesame 0x4c49462 thru 0x4c4bbfc data_size 0x1(0x279b bytes) 2024-09-16T11:13:48.824 INFO:teuthology.orchestra.run.smithi162.stdout:1844 trunc from 0xaedc0fc to 0x643de4d 2024-09-16T11:13:48.841 INFO:teuthology.orchestra.run.smithi162.stdout:1845 read 0x1404ef4 thru 0x141299a (0xdaa7 bytes) 2024-09-16T11:13:48.842 INFO:teuthology.orchestra.run.smithi162.stdout:1846 write 0x10d455b thru 0x10dec0f (0xa6b5 bytes) 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50255 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.850 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68783129099999996, 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063800420999999996, 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50255 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159697+0000", 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2e-07 2024-09-16T11:13:48.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159699+0000", 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.114e-06 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159705+0000", 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9970000000000004e-06 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159709+0000", 2024-09-16T11:13:48.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223159+0000", 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063450079000000006 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223206+0000", 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7148000000000003e-05 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223497+0000", 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000290434 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68781470700000003, 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.855 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:13:48.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68777965299999999, 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:13:48.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:48.864 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68776321900000004, 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:13:48.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68774691099999996, 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.869 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:13:48.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:48.873 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68773134499999999, 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:13:48.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.877 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68699685099999996, 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:13:48.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:13:48.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.882 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68698082299999996, 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68697099299999997, 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:13:48.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68366079700000004, 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:13:48.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:13:48.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68363744699999995, 2024-09-16T11:13:48.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:13:48.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.900 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68361814700000001, 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.901 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:13:48.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.905 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68360066500000005, 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:13:48.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.909 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68357008600000002, 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.910 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.683448269, 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:13:48.914 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:13:48.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:13:48.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68300709900000001, 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:13:48.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:48.923 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.682376227, 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:13:48.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68234794499999996, 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68233271699999998, 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.933 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:48.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.68231981500000005, 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:48.937 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:48.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:13:48.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:48.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:48.942 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:48.943 INFO:teuthology.orchestra.run.smithi162.stdout:1848 writesame 0x71103fd thru 0x711a24c data_size 0x44(0x9e50 bytes) 2024-09-16T11:13:48.943 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:48.943 INFO:teuthology.orchestra.run.smithi162.stdout:1850 read 0x6d721ec thru 0x6d7b59e (0x93b3 bytes) 2024-09-16T11:13:48.943 INFO:teuthology.orchestra.run.smithi162.stdout:1852 read 0x50bc29f thru 0x50c7578 (0xb2da bytes) 2024-09-16T11:13:48.943 INFO:teuthology.orchestra.run.smithi162.stdout:1853 writesame 0x38830a thru 0x3885da data_size 0x1(0x2d1 bytes) 2024-09-16T11:13:48.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:48.945 INFO:teuthology.orchestra.run.smithi162.stdout:1855 read 0x8325be thru 0x836284 (0x3cc7 bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1858 trunc from 0x711a24d to 0xccb121f 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1859 read 0x86629f9 thru 0x8664bc2 (0x21ca bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1865 write 0x3e5e9c thru 0x3e84cd (0x2632 bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1866 writesame 0x53a395b thru 0x53a82ef data_size 0x5b(0x4995 bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1867 trunc from 0xccb121f to 0xb22c1be 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1869 write 0x213d8ad thru 0x213dd4b (0x49f bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1870 punch from 0x23d845e to 0x23dabbd, (0x275f bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1871 writesame 0x2ee8250 thru 0x2eeddd1 data_size 0x11(0x5b82 bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1873 write 0x8ae2ae9 thru 0x8aebb69 (0x9081 bytes) 2024-09-16T11:13:48.946 INFO:teuthology.orchestra.run.smithi162.stdout:1874 writesame 0x34c7576 thru 0x34d5d49 data_size 0x4a(0xe7d4 bytes) 2024-09-16T11:13:48.947 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:48.937+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:48.957 INFO:teuthology.orchestra.run.smithi162.stdout:1875 punch from 0x143a12a to 0x1448914, (0xe7ea bytes) 2024-09-16T11:13:48.973 INFO:teuthology.orchestra.run.smithi162.stdout:1876 writesame 0xcdc4083 thru 0xcdca6a4 data_size 0x11(0x6622 bytes) 2024-09-16T11:13:48.973 INFO:teuthology.orchestra.run.smithi162.stdout:1878 writesame 0x7532d42 thru 0x75339fe data_size 0x1(0xcbd bytes) 2024-09-16T11:13:48.986 INFO:teuthology.orchestra.run.smithi162.stdout:1879 trunc from 0xcdca6a5 to 0xa4469d8 2024-09-16T11:13:49.000 INFO:teuthology.orchestra.run.smithi162.stdout:1880 read 0x4e77a50 thru 0x4e7f9ab (0x7f5c bytes) 2024-09-16T11:13:49.001 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:49.001 INFO:teuthology.orchestra.run.smithi162.stdout:1882 writesame 0x9d65c82 thru 0x9d6a262 data_size 0x3(0x45e1 bytes) 2024-09-16T11:13:49.018 INFO:teuthology.orchestra.run.smithi162.stdout:1883 write 0x2f89372 thru 0x2f8b1d7 (0x1e66 bytes) 2024-09-16T11:13:49.037 INFO:teuthology.orchestra.run.smithi162.stdout:1884 write 0xb90e886 thru 0xb91b50a (0xcc85 bytes) 2024-09-16T11:13:49.044 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:49.037+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:49.059 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:49.059 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:49.059 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:49.059 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:49.068 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:49.069 INFO:teuthology.orchestra.run.smithi162.stdout:1886 writesame 0xc7c0795 thru 0xc7c23b5 data_size 0x1(0x1c21 bytes) 2024-09-16T11:13:49.069 INFO:teuthology.orchestra.run.smithi162.stdout:1887 read 0x58dc343 thru 0x58ea85c (0xe51a bytes) 2024-09-16T11:13:49.069 INFO:teuthology.orchestra.run.smithi162.stdout:1888 writesame 0x757c502 thru 0x757d25f data_size 0x1d(0xd5e bytes) 2024-09-16T11:13:49.090 INFO:teuthology.orchestra.run.smithi162.stdout:1889 write 0xd66e77e thru 0xd66ff13 (0x1796 bytes) 2024-09-16T11:13:49.096 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:49.096 INFO:teuthology.orchestra.run.smithi162.stdout:1891 read 0x877658 thru 0x87a7b8 (0x3161 bytes) 2024-09-16T11:13:49.097 INFO:teuthology.orchestra.run.smithi162.stdout:1892 read 0xb8dffe thru 0xb8e426 (0x429 bytes) 2024-09-16T11:13:49.098 INFO:teuthology.orchestra.run.smithi162.stdout:1893 punch from 0xc904a61 to 0xc9075ea, (0x2b89 bytes) 2024-09-16T11:13:49.098 INFO:teuthology.orchestra.run.smithi162.stdout:1894 clone 15 order 19 su 65536 sc 12 2024-09-16T11:13:49.144 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:49.137+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:49.165 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:49.165 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:49.165 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:49.165 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:49.165 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:49.174 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:49.246 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:49.240+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:49.321 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 54.489919641999997, 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:49.322 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:49.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:49.327 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.199344871999998, 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.328 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:49.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.190936583999999, 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:49.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.340 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:49.342 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:49.361 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:13:49.365 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:49.360+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:49.440 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:49.440 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:49.440 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:49.440 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:49.447 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:49.440+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:49.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:49.456 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:49.502 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone14 from 0xd66ff14 (overlap 0xb35f7b) to 0x11eff2 2024-09-16T11:13:49.548 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:49.540+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:49.571 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:49.572 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:49.572 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:49.581 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:49.649 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-16T11:13:49.649 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:49.644+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:49.664 INFO:teuthology.orchestra.run.smithi162.stdout:1896 write 0x657f170 thru 0x658693d (0x77ce bytes) 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49775 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1286144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:49.682 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5444371270000001, 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050091745, 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49775 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:49.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:49.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133156+0000", 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1179999999999999e-06 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133159+0000", 2024-09-16T11:13:49.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133178+0000", 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8692e-05 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.182940+0000", 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049762038000000001 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183007+0000", 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7082000000000006e-05 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183247+0000", 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239581 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5444121910000002, 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:13:49.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:13:49.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.693 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.544278448, 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.694 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:13:49.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:13:49.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.54426006, 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:13:49.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:13:49.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:13:49.704 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5439593029999998, 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:13:49.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:13:49.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5439295980000001, 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:13:49.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5439162, 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:13:49.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.543902396, 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:13:49.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5435273970000001, 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:13:49.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:13:49.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50035 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5299990889999999, 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049942324000000003, 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50035 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:49.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147595+0000", 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9769999999999999e-06 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147601+0000", 2024-09-16T11:13:49.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147605+0000", 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197156+0000", 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049550679 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197231+0000", 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5853999999999996e-05 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197535+0000", 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000303861 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.734 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.529983366, 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:13:49.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5299617919999999, 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:13:49.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:13:49.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5299452929999999, 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.744 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5299292169999998, 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.749 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:49.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:13:49.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:13:49.753 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5295456430000001, 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:13:49.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50291 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.51750757, 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049900104000000001, 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50291 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160085+0000", 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7000000000000004e-07 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160089+0000", 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6679999999999999e-06 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160100+0000", 2024-09-16T11:13:49.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1311999999999999e-05 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160105+0000", 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.583e-06 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209651+0000", 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049546305999999998 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209730+0000", 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9295999999999994e-05 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209985+0000", 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254169 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50292 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.762 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5174827450000001, 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050216150000000001, 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50292 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:13:49.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160110+0000", 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160117+0000", 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.362e-06 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160124+0000", 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160126+0000", 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210014+0000", 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049887881000000002 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210108+0000", 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3356000000000005e-05 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210325+0000", 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021782400000000001 2024-09-16T11:13:49.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5174634660000001, 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.767 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:13:49.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5174552650000002, 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:49.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:49.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:13:49.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:13:49.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:13:49.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:13:49.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:13:49.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:13:49.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:13:49.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:13:49.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:13:49.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:13:49.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:13:49.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:13:49.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:13:49.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.780 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.781 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:49.781 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:49.781 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5174472429999999, 2024-09-16T11:13:49.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:13:49.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:49.782 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:49.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:49.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:49.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:49.782 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:13:49.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.783 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:49.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:49.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:49.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:49.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:49.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:49.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:49.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:49.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:13:49.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:49.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:13:49.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:13:49.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:49.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:13:49.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:13:49.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:49.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:13:49.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:13:49.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:49.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:13:49.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:13:49.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:49.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:13:49.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:13:49.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:49.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:49.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:49.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:13:49.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:13:49.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.791 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:49.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:49.792 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:49.793 INFO:teuthology.orchestra.run.smithi162.stdout:1897 writesame 0x88bd62c thru 0x88bd6b8 data_size 0x1(0x8d bytes) 2024-09-16T11:13:49.793 INFO:teuthology.orchestra.run.smithi162.stdout:1898 read 0x5f3840a thru 0x5f3c22d (0x3e24 bytes) 2024-09-16T11:13:49.795 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:49.796 INFO:teuthology.orchestra.run.smithi162.stdout:1899 read 0x8f8d2b1 thru 0x8f8dbd0 (0x920 bytes) 2024-09-16T11:13:49.796 INFO:teuthology.orchestra.run.smithi162.stdout:1900 punch from 0x509104e to 0x509ddd7, (0xcd89 bytes) 2024-09-16T11:13:49.796 INFO:teuthology.orchestra.run.smithi162.stdout:1902 trunc from 0xd66ff14 to 0x170d4ff 2024-09-16T11:13:49.797 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:13:49.760+0000 7f12a1290700 0 --2- 172.21.15.162:0/1546910189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f129c06d6f0 0x7f129c117660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T11:13:49.797 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:49.799 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:49.792+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:49.813 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:49.820 INFO:teuthology.orchestra.run.smithi162.stdout:1904 writesame 0xa157a0a thru 0xa15fe0e data_size 0x1(0x8405 bytes) 2024-09-16T11:13:49.820 INFO:teuthology.orchestra.run.smithi162.stdout:1905 punch from 0x7e67965 to 0x7e7197d, (0xa018 bytes) 2024-09-16T11:13:49.820 INFO:teuthology.orchestra.run.smithi162.stdout:1906 punch from 0x8ef3785 to 0x8efc4cd, (0x8d48 bytes) 2024-09-16T11:13:49.820 INFO:teuthology.orchestra.run.smithi162.stdout:1907 punch from 0x115b7a5 to 0x115e469, (0x2cc4 bytes) 2024-09-16T11:13:49.820 INFO:teuthology.orchestra.run.smithi162.stdout:1908 read 0x9363981 thru 0x936b36f (0x79ef bytes) 2024-09-16T11:13:49.820 INFO:teuthology.orchestra.run.smithi162.stdout:1909 punch from 0x74cc583 to 0x74d4ba7, (0x8624 bytes) 2024-09-16T11:13:49.821 INFO:teuthology.orchestra.run.smithi162.stdout:1910 writesame 0x6eb495d thru 0x6eb5257 data_size 0xb(0x8fb bytes) 2024-09-16T11:13:49.826 INFO:teuthology.orchestra.run.smithi162.stdout:1911 punch from 0x6f65904 to 0x6f667ed, (0xee9 bytes) 2024-09-16T11:13:49.826 INFO:teuthology.orchestra.run.smithi162.stdout:1912 read 0x345eebb thru 0x3469549 (0xa68f bytes) 2024-09-16T11:13:49.827 INFO:teuthology.orchestra.run.smithi162.stdout:1913 read 0x6bf3f5e thru 0x6c01206 (0xd2a9 bytes) 2024-09-16T11:13:49.827 INFO:teuthology.orchestra.run.smithi162.stdout:1914 write 0x3120c0d thru 0x312992b (0x8d1f bytes) 2024-09-16T11:13:49.831 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:49.832 INFO:teuthology.orchestra.run.smithi162.stdout:1915 read 0xa135ed1 thru 0xa1397b0 (0x38e0 bytes) 2024-09-16T11:13:49.842 INFO:teuthology.orchestra.run.smithi162.stdout:1916 writesame 0xa4f3510 thru 0xa4f88bc data_size 0x1(0x53ad bytes) 2024-09-16T11:13:49.852 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:49.852 INFO:teuthology.orchestra.run.smithi162.stdout:1917 trunc from 0xa4f88bd to 0x3fdc9af 2024-09-16T11:13:49.873 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:49.874 INFO:teuthology.orchestra.run.smithi162.stdout:1919 clone 16 order 19 su 8192 sc 12 2024-09-16T11:13:49.894 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:49.899 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:49.893+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:49.908 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:49.908 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:49.917 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:50.001 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:49.993+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:50.016 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.016 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.026 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:50.102 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:50.093+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:50.125 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:50.149 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:50.152 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:50.152 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:50.152 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:50.153 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:50.153 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:50.153 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:50.153 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:50.153 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:50.168 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.168 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:50.203 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:50.196+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:50.272 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.272 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.281 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:50.304 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:50.297+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:50.394 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.394 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.405 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:50.405 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:50.396+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:50.501 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.501 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.506 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:50.497+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:50.510 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:50.537 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone15 from 0x3fdc9af (overlap 0x170d4ff) to 0x6ea64c 2024-09-16T11:13:50.607 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:50.601+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:50.630 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.630 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.642 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:50.661 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:50.661 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:50.661 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T11:13:50.661 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.364303 2024-09-16T11:13:50.662 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.36430318739869705 2024-09-16T11:13:50.707 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:50.701+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:50.794 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.795 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.796 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-16T11:13:50.804 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:50.808 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:50.800+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:50.817 INFO:teuthology.orchestra.run.smithi162.stdout:1921 read 0x1ea452e thru 0x1eb0b06 (0xc5d9 bytes) 2024-09-16T11:13:50.856 INFO:teuthology.orchestra.run.smithi162.stdout:1922 write 0xe716291 thru 0xe725feb (0xfd5b bytes) 2024-09-16T11:13:50.870 INFO:teuthology.orchestra.run.smithi162.stdout:1924 read 0x2f90a66 thru 0x2f97c11 (0x71ac bytes) 2024-09-16T11:13:50.870 INFO:teuthology.orchestra.run.smithi162.stdout:1925 writesame 0xbc8b01b thru 0xbc93a4d data_size 0x3(0x8a33 bytes) 2024-09-16T11:13:50.890 INFO:teuthology.orchestra.run.smithi162.stdout:1927 read 0x50877ca thru 0x5097733 (0xff6a bytes) 2024-09-16T11:13:50.890 INFO:teuthology.orchestra.run.smithi162.stdout:1928 read 0xa082091 thru 0xa08cae6 (0xaa56 bytes) 2024-09-16T11:13:50.890 INFO:teuthology.orchestra.run.smithi162.stdout:1929 trunc from 0xe725fec to 0xe7fb731 2024-09-16T11:13:50.899 INFO:teuthology.orchestra.run.smithi162.stdout:1930 write 0xa0b917e thru 0xa0c65ae (0xd431 bytes) 2024-09-16T11:13:50.905 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:50.905 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:50.908 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:50.901+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:50.915 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:50.917 INFO:teuthology.orchestra.run.smithi162.stdout:1931 write 0xa5d2750 thru 0xa5d914c (0x69fd bytes) 2024-09-16T11:13:50.927 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:50.927 INFO:teuthology.orchestra.run.smithi162.stdout:1933 write 0x4b4438c thru 0x4b50e2a (0xca9f bytes) 2024-09-16T11:13:50.942 INFO:teuthology.orchestra.run.smithi162.stdout:1934 read 0x8d219b thru 0x8e1d20 (0xfb86 bytes) 2024-09-16T11:13:50.949 INFO:teuthology.orchestra.run.smithi162.stdout:1935 read 0x139c66 thru 0x13bcf7 (0x2092 bytes) 2024-09-16T11:13:51.010 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:51.001+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:51.049 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.049 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.058 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:51.111 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:51.108+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:51.151 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.151 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.163 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:51.170 INFO:teuthology.orchestra.run.smithi162.stdout:1936 write 0x2a905a3 thru 0x2a9687d (0x62db bytes) 2024-09-16T11:13:51.201 INFO:teuthology.orchestra.run.smithi162.stdout:1937 trunc from 0xe7fb731 to 0x7eb7ae9 2024-09-16T11:13:51.211 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:51.204+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:51.221 INFO:teuthology.orchestra.run.smithi162.stdout:1939 read 0x6f4819c thru 0x6f4f35d (0x71c2 bytes) 2024-09-16T11:13:51.221 INFO:teuthology.orchestra.run.smithi162.stdout:1940 trunc from 0x7eb7ae9 to 0x81b9f08 2024-09-16T11:13:51.228 INFO:teuthology.orchestra.run.smithi162.stdout:1941 punch from 0xb67209 to 0xb72466, (0xb25d bytes) 2024-09-16T11:13:51.229 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:51.229 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:51.229 INFO:teuthology.orchestra.run.smithi162.stdout:1944 read 0x1831c36 thru 0x183396c (0x1d37 bytes) 2024-09-16T11:13:51.247 INFO:teuthology.orchestra.run.smithi162.stdout:1945 write 0xd45992a thru 0xd45cc86 (0x335d bytes) 2024-09-16T11:13:51.260 INFO:teuthology.orchestra.run.smithi162.stdout:1946 trunc from 0xd45cc87 to 0xed7c23b 2024-09-16T11:13:51.263 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.263 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.269 INFO:teuthology.orchestra.run.smithi162.stdout:1947 read 0x13e62a3 thru 0x13e8fb1 (0x2d0f bytes) 2024-09-16T11:13:51.272 INFO:teuthology.orchestra.run.smithi162.stdout:1948 punch from 0x8e4c1de to 0x8e53904, (0x7726 bytes) 2024-09-16T11:13:51.274 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:51.275 INFO:teuthology.orchestra.run.smithi162.stdout:1950 write 0x70d3cb3 thru 0x70d8b25 (0x4e73 bytes) 2024-09-16T11:13:51.311 INFO:teuthology.orchestra.run.smithi162.stdout:1951 trunc from 0xed7c23b to 0x1b6bab6 2024-09-16T11:13:51.312 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:51.304+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:51.376 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:13:51.389 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.1 primary-affinity to 0.364303 (8238742) 2024-09-16T11:13:51.392 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.392 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.393 INFO:teuthology.orchestra.run.smithi162.stdout:1952 write 0xd05f2c8 thru 0xd066e82 (0x7bbb bytes) 2024-09-16T11:13:51.397 INFO:teuthology.orchestra.run.smithi162.stdout:1954 read 0x75ed767 thru 0x75ef86f (0x2109 bytes) 2024-09-16T11:13:51.397 INFO:teuthology.orchestra.run.smithi162.stdout:1960 trunc from 0xd066e83 to 0xd531bc9 2024-09-16T11:13:51.401 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:51.405 INFO:teuthology.orchestra.run.smithi162.stdout:1961 write 0x1c4f048 thru 0x1c567bf (0x7778 bytes) 2024-09-16T11:13:51.410 INFO:teuthology.orchestra.run.smithi162.stdout:1963 write 0x37bd2cc thru 0x37cc1c6 (0xeefb bytes) 2024-09-16T11:13:51.412 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:51.404+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:51.431 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:51.431 INFO:teuthology.orchestra.run.smithi162.stdout:1965 punch from 0xe1615a to 0xe1fd0e, (0x9bb4 bytes) 2024-09-16T11:13:51.431 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:51.432 INFO:teuthology.orchestra.run.smithi162.stdout:1967 writesame 0xb239214 thru 0xb245b58 data_size 0x4b(0xc945 bytes) 2024-09-16T11:13:51.446 INFO:teuthology.orchestra.run.smithi162.stdout:1968 writesame 0xa7de329 thru 0xa7e33ec data_size 0xc(0x50c4 bytes) 2024-09-16T11:13:51.453 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:51.453 INFO:teuthology.orchestra.run.smithi162.stdout:1970 writesame 0x8ec57ee thru 0x8eca0c2 data_size 0xf(0x48d5 bytes) 2024-09-16T11:13:51.465 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:51.465 INFO:teuthology.orchestra.run.smithi162.stdout:1976 write 0xc9cd544 thru 0xc9d0640 (0x30fd bytes) 2024-09-16T11:13:51.508 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.509 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.514 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:51.505+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:51.518 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:51.615 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:51.613+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:51.660 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.660 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.669 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:51.715 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:51.708+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:51.768 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.769 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.777 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:51.816 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:51.813+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:51.917 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:51.912+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:51.923 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:51.924 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:51.933 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:52.018 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:52.012+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:52.028 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:52.028 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:52.038 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:52.119 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:52.113+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:52.134 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:52.134 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:52.143 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:52.165 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:52.219 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:52.212+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:52.262 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:52.271 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:52.307 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:52.327 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:52.327 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:52.320+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:52.350 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:52.367 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:52.390 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:13:52.392 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:52.395 INFO:teuthology.orchestra.run.smithi162.stdout:1977 write 0x77c2de5 thru 0x77cf114 (0xc330 bytes) 2024-09-16T11:13:52.409 INFO:teuthology.orchestra.run.smithi162.stdout:1978 write 0x27589a thru 0x276ee3 (0x164a bytes) 2024-09-16T11:13:52.421 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:52.421 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:52.413+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:52.422 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:52.422 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:52.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:52.422 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:52.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.590701545000002, 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:52.423 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:52.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:52.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:52.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:52.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:52.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:52.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:52.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.300126775000003, 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:52.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:52.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:52.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:52.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:52.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:52.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:52.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:52.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:52.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:52.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:52.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:52.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:52.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:52.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.291718486999997, 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:52.441 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:52.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:52.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:52.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.447 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:52.448 INFO:teuthology.orchestra.run.smithi162.stdout:1980 write 0x2d187d8 thru 0x2d19183 (0x9ac bytes) 2024-09-16T11:13:52.449 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:1981 read 0x630a9fe thru 0x6316f35 (0xc538 bytes) 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:1986 read 0x3de3832 thru 0x3deacf0 (0x74bf bytes) 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:1987 read 0x4dc3f3a thru 0x4dd1cb3 (0xdd7a bytes) 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:1989 read 0x55b0042 thru 0x55b7942 (0x7901 bytes) 2024-09-16T11:13:52.451 INFO:teuthology.orchestra.run.smithi162.stdout:1990 write 0x697dc2a thru 0x697e486 (0x85d bytes) 2024-09-16T11:13:52.452 INFO:teuthology.orchestra.run.smithi162.stdout:1991 write 0x2ea7ee3 thru 0x2eb1048 (0x9166 bytes) 2024-09-16T11:13:52.452 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:52.456 INFO:teuthology.orchestra.run.smithi162.stdout:1992 write 0x14a5bf7 thru 0x14b4886 (0xec90 bytes) 2024-09-16T11:13:52.478 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:52.481 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:52.481 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:52.481 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:52.481 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:52.481 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:52.482 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:52.482 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:52.482 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:52.492 INFO:teuthology.orchestra.run.smithi162.stdout:1993 read 0x9a5e2c5 thru 0x9a6c530 (0xe26c bytes) 2024-09-16T11:13:52.492 INFO:teuthology.orchestra.run.smithi162.stdout:1994 write 0x96b4760 thru 0x96c3956 (0xf1f7 bytes) 2024-09-16T11:13:52.522 INFO:teuthology.orchestra.run.smithi162.stdout:1995 trunc from 0xd531bc9 to 0x29440dc 2024-09-16T11:13:52.522 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:52.517+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:52.550 INFO:teuthology.orchestra.run.smithi162.stdout:1996 punch from 0x70701a to 0x70e3ec, (0x73d2 bytes) 2024-09-16T11:13:52.569 INFO:teuthology.orchestra.run.smithi162.stdout:1997 write 0x8e0dc39 thru 0x8e16857 (0x8c1f bytes) 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:55965 4.17 4:eb5f5625:::rbd_object_map.11b5624fb08c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:52.568233+0000", 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00053820400000000003, 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00057323900000000002, 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.570 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 55965 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568233+0000", 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568233+0000", 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568235+0000", 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6819999999999999e-06 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568243+0000", 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0139999999999998e-06 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568266+0000", 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2702999999999999e-05 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568275+0000", 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0319999999999992e-06 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568373+0000", 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8400999999999995e-05 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568528+0000", 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015463200000000001 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:55966 4.17 4.a46afad7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:52.568340+0000", 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.000431932, 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054452000000000005, 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 55966 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568340+0000", 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568340+0000", 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568341+0000", 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.406e-06 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568348+0000", 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4089999999999997e-06 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568365+0000", 2024-09-16T11:13:52.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7044000000000001e-05 2024-09-16T11:13:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568375+0000", 2024-09-16T11:13:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4350000000000003e-06 2024-09-16T11:13:52.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:55967 4.17 4.a46afad7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:52.568388+0000", 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00038337799999999997, 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055204699999999995, 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 55967 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568388+0000", 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568388+0000", 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568389+0000", 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5799999999999998e-07 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568393+0000", 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:13:52.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568400+0000", 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.092e-06 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568404+0000", 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5290000000000002e-06 2024-09-16T11:13:52.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:55968 4.17 4.a46afad7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:52.568411+0000", 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.000360832, 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058261299999999995, 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:52.586 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 55968 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568411+0000", 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568411+0000", 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568411+0000", 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1999999999999999e-07 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568415+0000", 2024-09-16T11:13:52.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6179999999999999e-06 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568421+0000", 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3509999999999998e-06 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568425+0000", 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7390000000000001e-06 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:55969 4.17 4.a46afad7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:52.568429+0000", 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.000342374, 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00062994099999999996, 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 55969 2024-09-16T11:13:52.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568429+0000", 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568429+0000", 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568430+0000", 2024-09-16T11:13:52.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2799999999999996e-07 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568433+0000", 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.96e-06 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568438+0000", 2024-09-16T11:13:52.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9290000000000003e-06 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568442+0000", 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8330000000000004e-06 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.594 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:55970 4.17 4.a46afad7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:52.568453+0000", 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00031844399999999998, 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00065907900000000002, 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 55970 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568453+0000", 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568453+0000", 2024-09-16T11:13:52.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568454+0000", 2024-09-16T11:13:52.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6000000000000004e-07 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568457+0000", 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1520000000000001e-06 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568479+0000", 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2583e-05 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:52.568484+0000", 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.442e-06 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:52.600 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 6 2024-09-16T11:13:52.600 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:52.601 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:52.601 INFO:teuthology.orchestra.run.smithi162.stdout:1998 write 0x2a6fee2 thru 0x2a79fba (0xa0d9 bytes) 2024-09-16T11:13:52.605 INFO:teuthology.orchestra.run.smithi162.stdout:2001 read 0x1f4939d thru 0x1f50476 (0x70da bytes) 2024-09-16T11:13:52.605 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.605 INFO:teuthology.orchestra.run.smithi162.stdout:2004 read 0x693fb90 thru 0x69413e1 (0x1852 bytes) 2024-09-16T11:13:52.605 INFO:teuthology.orchestra.run.smithi162.stdout:2005 punch from 0x8bd1ede to 0x8bd27e0, (0x902 bytes) 2024-09-16T11:13:52.607 INFO:teuthology.orchestra.run.smithi162.stdout:2006 trunc from 0x8e16858 to 0x9ffa2eb 2024-09-16T11:13:52.617 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.617 INFO:teuthology.orchestra.run.smithi162.stdout:2008 punch from 0x9171ba6 to 0x9181497, (0xf8f1 bytes) 2024-09-16T11:13:52.625 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:52.617+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:52.637 INFO:teuthology.orchestra.run.smithi162.stdout:2009 write 0xdcc9988 thru 0xdcd9668 (0xfce1 bytes) 2024-09-16T11:13:52.657 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.658 INFO:teuthology.orchestra.run.smithi162.stdout:2014 read 0x537cef5 thru 0x538a9ca (0xdad6 bytes) 2024-09-16T11:13:52.677 INFO:teuthology.orchestra.run.smithi162.stdout:2015 trunc from 0xdcd9669 to 0x1c9e7c6 2024-09-16T11:13:52.701 INFO:teuthology.orchestra.run.smithi162.stdout:2016 punch from 0x12d8374 to 0x12dca59, (0x46e5 bytes) 2024-09-16T11:13:52.718 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:52.718 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:52.718 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:52.718 INFO:teuthology.orchestra.run.smithi162.stdout:2017 write 0x6e88dfb thru 0x6e95c0a (0xce10 bytes) 2024-09-16T11:13:52.725 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:52.720+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:52.727 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:52.755 INFO:teuthology.orchestra.run.smithi162.stdout:2020 write 0x9dccae7 thru 0x9dd5fb3 (0x94cd bytes) 2024-09-16T11:13:52.769 INFO:teuthology.orchestra.run.smithi162.stdout:2021 write 0xbd3bb6 thru 0xbdb550 (0x799b bytes) 2024-09-16T11:13:52.771 INFO:teuthology.orchestra.run.smithi162.stdout:2022 writesame 0x684054c thru 0x6850252 data_size 0x5(0xfd07 bytes) 2024-09-16T11:13:52.807 INFO:teuthology.orchestra.run.smithi162.stdout:2024 writesame 0xe4a805c thru 0xe4b0809 data_size 0x15(0x87ae bytes) 2024-09-16T11:13:52.809 INFO:teuthology.orchestra.run.smithi162.stdout:2025 writesame 0x77897a thru 0x779489 data_size 0x10(0xb10 bytes) 2024-09-16T11:13:52.811 INFO:teuthology.orchestra.run.smithi162.stdout:2028 writesame 0x3ecd087 thru 0x3ed5adb data_size 0x1(0x8a55 bytes) 2024-09-16T11:13:52.829 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:52.820+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:52.834 INFO:teuthology.orchestra.run.smithi162.stdout:2029 trunc from 0xe4b080a to 0xebdb55f 2024-09-16T11:13:52.844 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.844 INFO:teuthology.orchestra.run.smithi162.stdout:2031 read 0xda3a9f8 thru 0xda43958 (0x8f61 bytes) 2024-09-16T11:13:52.844 INFO:teuthology.orchestra.run.smithi162.stdout:2032 punch from 0x8e0f745 to 0x8e189db, (0x9296 bytes) 2024-09-16T11:13:52.844 INFO:teuthology.orchestra.run.smithi162.stdout:2033 read 0xc036d1e thru 0xc041efa (0xb1dd bytes) 2024-09-16T11:13:52.844 INFO:teuthology.orchestra.run.smithi162.stdout:2034 punch from 0xde31db to 0xdf1412, (0xe237 bytes) 2024-09-16T11:13:52.863 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:52.863 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49775 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1286144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.726362312, 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050091745, 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.864 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49775 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133156+0000", 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1179999999999999e-06 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133159+0000", 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:13:52.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133178+0000", 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8692e-05 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.182940+0000", 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049762038000000001 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183007+0000", 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7082000000000006e-05 2024-09-16T11:13:52.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183247+0000", 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239581 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.726337376, 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:13:52.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:13:52.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7262036329999999, 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.877 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7261852449999999, 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:52.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7258844880000002, 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:13:52.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7258547829999999, 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.886 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:13:52.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7258413849999998, 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:13:52.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7258275809999999, 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.725452582, 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:13:52.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50035 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:52.903 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7119242740000002, 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049942324000000003, 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50035 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147595+0000", 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9769999999999999e-06 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147601+0000", 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147605+0000", 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197156+0000", 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049550679 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197231+0000", 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5853999999999996e-05 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197535+0000", 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000303861 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.907 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7119085509999996, 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:13:52.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7118869769999998, 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.916 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7118704779999998, 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:13:52.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7118544020000002, 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.921 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:13:52.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.7114708280000004, 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.926 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:13:52.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:13:52.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.929 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50291 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6994327550000001, 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049900104000000001, 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50291 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160085+0000", 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7000000000000004e-07 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160089+0000", 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6679999999999999e-06 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160100+0000", 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1311999999999999e-05 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160105+0000", 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.583e-06 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209651+0000", 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049546305999999998 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209730+0000", 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9295999999999994e-05 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209985+0000", 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254169 2024-09-16T11:13:52.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50292 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6994079299999996, 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050216150000000001, 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.934 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50292 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160110+0000", 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160117+0000", 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.362e-06 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160124+0000", 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160126+0000", 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210014+0000", 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049887881000000002 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210108+0000", 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3356000000000005e-05 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210325+0000", 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021782400000000001 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6993886509999996, 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:13:52.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:13:52.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6993804499999996, 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:13:52.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:52.946 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6993724280000002, 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:13:52.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:52.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:52.951 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:52.951 INFO:teuthology.orchestra.run.smithi162.stdout:2035 trunc from 0xebdb55f to 0xd9920c 2024-09-16T11:13:52.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:52.953 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.953 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:52.953 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:52.944+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:52.965 INFO:teuthology.orchestra.run.smithi162.stdout:2038 trunc from 0xd9920c to 0x96451b6 2024-09-16T11:13:52.976 INFO:teuthology.orchestra.run.smithi162.stdout:2039 punch from 0x3addfd7 to 0x3adf1ed, (0x1216 bytes) 2024-09-16T11:13:52.976 INFO:teuthology.orchestra.run.smithi162.stdout:2040 write 0x58a91fc thru 0x58a96fb (0x500 bytes) 2024-09-16T11:13:52.989 INFO:teuthology.orchestra.run.smithi162.stdout:2041 trunc from 0x96451b6 to 0x64ca520 2024-09-16T11:13:53.027 INFO:teuthology.orchestra.run.smithi162.stdout:2042 writesame 0xecdad65 thru 0xecde003 data_size 0x1(0x329f bytes) 2024-09-16T11:13:53.028 INFO:teuthology.orchestra.run.smithi162.stdout:2045 write 0x13a55c5 thru 0x13af786 (0xa1c2 bytes) 2024-09-16T11:13:53.042 INFO:teuthology.orchestra.run.smithi162.stdout:2046 write 0xa65b6b1 thru 0xa6672f1 (0xbc41 bytes) 2024-09-16T11:13:53.054 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:53.045+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:53.060 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.060 INFO:teuthology.orchestra.run.smithi162.stdout:2048 write 0x2acff06 thru 0x2addb09 (0xdc04 bytes) 2024-09-16T11:13:53.073 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:53.073 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:53.073 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:53.073 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:53.079 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.079 INFO:teuthology.orchestra.run.smithi162.stdout:2050 read 0x7343ce5 thru 0x734e7e4 (0xab00 bytes) 2024-09-16T11:13:53.079 INFO:teuthology.orchestra.run.smithi162.stdout:2052 write 0xe76aed3 thru 0xe77503d (0xa16b bytes) 2024-09-16T11:13:53.085 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:53.092 INFO:teuthology.orchestra.run.smithi162.stdout:2053 write 0x8be028f thru 0x8be665c (0x63ce bytes) 2024-09-16T11:13:53.104 INFO:teuthology.orchestra.run.smithi162.stdout:2054 punch from 0x7a222f to 0x7af3c3, (0xd194 bytes) 2024-09-16T11:13:53.104 INFO:teuthology.orchestra.run.smithi162.stdout:2055 write 0xd46ebc2 thru 0xd47cbf5 (0xe034 bytes) 2024-09-16T11:13:53.122 INFO:teuthology.orchestra.run.smithi162.stdout:2056 read 0x20fe163 thru 0x2103f9e (0x5e3c bytes) 2024-09-16T11:13:53.133 INFO:teuthology.orchestra.run.smithi162.stdout:2057 trunc from 0xecde004 to 0x95f61a0 2024-09-16T11:13:53.154 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:53.145+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:53.155 INFO:teuthology.orchestra.run.smithi162.stdout:2058 write 0x7585211 thru 0x758bb33 (0x6923 bytes) 2024-09-16T11:13:53.165 INFO:teuthology.orchestra.run.smithi162.stdout:2060 read 0x2f648c6 thru 0x2f69793 (0x4ece bytes) 2024-09-16T11:13:53.188 INFO:teuthology.orchestra.run.smithi162.stdout:2062 writesame 0xe0b739c thru 0xe0c0f5d data_size 0x1(0x9bc2 bytes) 2024-09-16T11:13:53.189 INFO:teuthology.orchestra.run.smithi162.stdout:2065 punch from 0xabed024 to 0xabf293c, (0x5918 bytes) 2024-09-16T11:13:53.189 INFO:teuthology.orchestra.run.smithi162.stdout:2066 read 0x459da8a thru 0x45a35fb (0x5b72 bytes) 2024-09-16T11:13:53.189 INFO:teuthology.orchestra.run.smithi162.stdout:2068 read 0xa2cf594 thru 0xa2d0af7 (0x1564 bytes) 2024-09-16T11:13:53.199 INFO:teuthology.orchestra.run.smithi162.stdout:2070 write 0xe1d80a4 thru 0xe1dea1b (0x6978 bytes) 2024-09-16T11:13:53.202 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:53.202 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:53.202 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:53.202 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:53.202 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:53.212 INFO:teuthology.orchestra.run.smithi162.stdout:2071 writesame 0x4e1cdba thru 0x4e1e649 data_size 0x18(0x1890 bytes) 2024-09-16T11:13:53.217 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:53.218 INFO:teuthology.orchestra.run.smithi162.stdout:2072 write 0xcb49a0d thru 0xcb4a47d (0xa71 bytes) 2024-09-16T11:13:53.224 INFO:teuthology.orchestra.run.smithi162.stdout:2073 read 0x956d257 thru 0x9578f31 (0xbcdb bytes) 2024-09-16T11:13:53.224 INFO:teuthology.orchestra.run.smithi162.stdout:2074 write 0x2f23cf5 thru 0x2f33585 (0xf891 bytes) 2024-09-16T11:13:53.243 INFO:teuthology.orchestra.run.smithi162.stdout:2075 trunc from 0xe1dea1c to 0xad1e5ad 2024-09-16T11:13:53.254 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:53.248+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:53.281 INFO:teuthology.orchestra.run.smithi162.stdout:2078 write 0xde0d3d0 thru 0xde12ab6 (0x56e7 bytes) 2024-09-16T11:13:53.296 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.296 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.296 INFO:teuthology.orchestra.run.smithi162.stdout:2082 writesame 0xc31f4c5 thru 0xc324fcf data_size 0x11(0x5b0b bytes) 2024-09-16T11:13:53.305 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.305 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.305 INFO:teuthology.orchestra.run.smithi162.stdout:2087 punch from 0x2c0e141 to 0x2c16974, (0x8833 bytes) 2024-09-16T11:13:53.305 INFO:teuthology.orchestra.run.smithi162.stdout:2089 punch from 0xb9b60c to 0xba3fd0, (0x89c4 bytes) 2024-09-16T11:13:53.326 INFO:teuthology.orchestra.run.smithi162.stdout:2090 trunc from 0xde12ab7 to 0x2808fdc 2024-09-16T11:13:53.351 INFO:teuthology.orchestra.run.smithi162.stdout:2091 writesame 0x77cacd thru 0x783550 data_size 0x22(0x6a84 bytes) 2024-09-16T11:13:53.356 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:53.348+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:53.375 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15969 4.d 4:b69afd92:::rbd_data.11b58a761a59.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.292442821000002, 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044609673000000002, 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15969 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079092+0000", 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.17e-06 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079104+0000", 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079140+0000", 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5648000000000002e-05 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079152+0000", 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2636999999999999e-05 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079270+0000", 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011812200000000001 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079499+0000", 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022818 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.082238+0000", 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027396949999999999 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123504+0000", 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041265556000000002 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123526+0000", 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1971999999999999e-05 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123623+0000", 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7347000000000002e-05 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123700+0000", 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6786000000000006e-05 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15970 4.d 4:b7b5bf19:::rbd_data.11b58a761a59.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.292138159, 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044492102999999998, 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15970 2024-09-16T11:13:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079397+0000", 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.793e-06 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079405+0000", 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079421+0000", 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6201e-05 2024-09-16T11:13:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079429+0000", 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.092788+0000", 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013358761 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.093013+0000", 2024-09-16T11:13:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000225687 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.098959+0000", 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059459059999999999 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123735+0000", 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024775755999999999 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123753+0000", 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8065999999999999e-05 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123836+0000", 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-05 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123887+0000", 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0559000000000002e-05 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:15969 4.d e47/36 ondisk, result = 0)", 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.281621490999999, 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033797389999999997, 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089913+0000", 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.04e-06 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089921+0000", 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2309999999999996e-06 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089941+0000", 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0494000000000001e-05 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089948+0000", 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4779999999999996e-06 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123470+0000", 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033522226000000002 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123488+0000", 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8269000000000001e-05 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123709+0000", 2024-09-16T11:13:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220652 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26656 4.9 4:95b731cd:::rbd_data.11b5499bda95.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.420633153000001, 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.054121295999999999, 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26656 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950902+0000", 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950914+0000", 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1724e-05 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950943+0000", 2024-09-16T11:13:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9621e-05 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950956+0000", 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.238e-05 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951090+0000", 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013472399999999999 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951475+0000", 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000384378 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.964699+0000", 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013224217999999999 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004806+0000", 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.040106941 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004820+0000", 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3909e-05 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004895+0000", 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4905000000000002e-05 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.005021+0000", 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012639300000000001 2024-09-16T11:13:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49320 4.1a 4:5c7f3453:::rbd_data.11b5eaebe1b2.0000000000000004:11 [read 1466368~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134586+0000", 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.236946627, 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029924813000000001, 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49320 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134586+0000", 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134586+0000", 2024-09-16T11:13:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134587+0000", 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4499999999999998e-07 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134588+0000", 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2500000000000001e-06 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134591+0000", 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134593+0000", 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1720000000000001e-06 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164197+0000", 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029604001000000001 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164256+0000", 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8882000000000003e-05 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164511+0000", 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025535 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49321 4.1a 4:5c7f3453:::rbd_data.11b5eaebe1b2.0000000000000004:11 [read 1531904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.236938136, 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030292619999999999, 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49321 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:13:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134596+0000", 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1480000000000001e-06 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134599+0000", 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.982e-06 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134601+0000", 2024-09-16T11:13:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9970000000000001e-06 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164533+0000", 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029932041 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164600+0000", 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6744999999999999e-05 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164888+0000", 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028745099999999998 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49361 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1007616~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134747+0000", 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367857799999999, 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029995797000000001, 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49361 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134747+0000", 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134747+0000", 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134748+0000", 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.67e-07 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134749+0000", 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.331e-06 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134752+0000", 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2830000000000002e-06 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134755+0000", 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164180+0000", 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029425617000000001 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164403+0000", 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022296100000000001 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164743+0000", 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00033982000000000001 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49362 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1073152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367763800000002, 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030314636999999998, 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49362 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4999999999999999e-07 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134758+0000", 2024-09-16T11:13:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2929999999999999e-06 2024-09-16T11:13:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134761+0000", 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134763+0000", 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.12e-06 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164759+0000", 2024-09-16T11:13:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029996030999999999 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164824+0000", 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4828999999999999e-05 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165071+0000", 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024724099999999999 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49363 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1138688~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367664090000003, 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030709633, 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49363 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:13:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5400000000000002e-07 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134768+0000", 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1880000000000001e-06 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134771+0000", 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.106e-06 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134774+0000", 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4339999999999999e-06 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165113+0000", 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030339123999999999 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165176+0000", 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3652999999999995e-05 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165476+0000", 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000299874 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49364 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1204224~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367574640000001, 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.03118348, 2024-09-16T11:13:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49364 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4699999999999998e-07 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134777+0000", 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134780+0000", 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8040000000000002e-06 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134782+0000", 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9269999999999999e-06 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165521+0000", 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030739417000000002 2024-09-16T11:13:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165610+0000", 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9164000000000002e-05 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165959+0000", 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034861899999999999 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49365 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367492369999997, 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.03122434, 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49365 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134785+0000", 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.17e-06 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134788+0000", 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8049999999999998e-06 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134790+0000", 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.193e-06 2024-09-16T11:13:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166000+0000", 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031209496999999999 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166008+0000", 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4239999999999993e-06 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49366 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367409709999997, 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031247255000000002, 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49366 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134794+0000", 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2500000000000001e-06 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134797+0000", 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9359999999999999e-06 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134799+0000", 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166029+0000", 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031230734999999999 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166039+0000", 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8959999999999998e-06 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49367 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367324850000001, 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0312546, 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49367 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:13:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134802+0000", 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1820000000000001e-06 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134805+0000", 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6960000000000001e-06 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134807+0000", 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.844e-06 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166051+0000", 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031244416000000001 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166055+0000", 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9160000000000003e-06 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49368 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367244729999998, 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031285238, 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49368 2024-09-16T11:13:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134810+0000", 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1629999999999999e-06 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134813+0000", 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.92e-06 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134815+0000", 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0609999999999998e-06 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166084+0000", 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031268351 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166094+0000", 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.022e-05 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49369 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2367160589999999, 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031292820999999998, 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49369 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:13:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134818+0000", 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-06 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134821+0000", 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:13:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134824+0000", 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166105+0000", 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031281010999999997 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166110+0000", 2024-09-16T11:13:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2730000000000002e-06 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49457 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 483328~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135181+0000", 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2363521930000001, 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029862390999999999, 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49457 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135181+0000", 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135181+0000", 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135182+0000", 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6700000000000002e-07 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135184+0000", 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135191+0000", 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5880000000000003e-06 2024-09-16T11:13:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135195+0000", 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0069999999999999e-06 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164917+0000", 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029721813 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164961+0000", 2024-09-16T11:13:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4740999999999997e-05 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165043+0000", 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1910999999999996e-05 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49460 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 679936~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135238+0000", 2024-09-16T11:13:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2362947389999999, 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029937043999999999, 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49460 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135238+0000", 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135238+0000", 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135239+0000", 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135241+0000", 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.373e-06 2024-09-16T11:13:53.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135247+0000", 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8259999999999998e-06 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135253+0000", 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7520000000000002e-06 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165064+0000", 2024-09-16T11:13:53.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029811212 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165114+0000", 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.939e-05 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1870000000000002e-05 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49463 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 876544~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135272+0000", 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2362606390000002, 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030058985999999999, 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49463 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.458 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135272+0000", 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135272+0000", 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135273+0000", 2024-09-16T11:13:53.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.67e-07 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135274+0000", 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6479999999999999e-06 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135300+0000", 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5609000000000002e-05 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135323+0000", 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2892999999999999e-05 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165194+0000", 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029871051999999999 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165249+0000", 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5052000000000001e-05 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165331+0000", 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2465000000000004e-05 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49466 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 1073152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135329+0000", 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2362039899999999, 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030139170999999999, 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49466 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135329+0000", 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135329+0000", 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135330+0000", 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0200000000000002e-07 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135333+0000", 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.461e-06 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135340+0000", 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4980000000000003e-06 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135346+0000", 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2160000000000003e-06 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165362+0000", 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030016571999999998 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165402+0000", 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9749999999999997e-05 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165468+0000", 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6072e-05 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49477 4.e 4.d5b60d6e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.466 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.136209+0000", 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.2353238170000003, 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029922181999999999, 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49477 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.136209+0000", 2024-09-16T11:13:53.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.136209+0000", 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.136210+0000", 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4399999999999998e-07 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.136215+0000", 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4900000000000002e-06 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.136236+0000", 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1328e-05 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.136242+0000", 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9170000000000003e-06 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166122+0000", 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029880164000000001 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166131+0000", 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3389999999999999e-06 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.470 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:53.473 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:53.473 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:53.465+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:53.474 INFO:teuthology.orchestra.run.smithi162.stdout:2092 punch from 0x8f3ad1 to 0x8f72b5, (0x37e4 bytes) 2024-09-16T11:13:53.474 INFO:teuthology.orchestra.run.smithi162.stdout:2093 writesame 0x19f7abd thru 0x19fe032 data_size 0x6f(0x6576 bytes) 2024-09-16T11:13:53.474 INFO:teuthology.orchestra.run.smithi162.stdout:2094 trunc from 0x2808fdc to 0xb6ab451 2024-09-16T11:13:53.474 INFO:teuthology.orchestra.run.smithi162.stdout:2095 writesame 0x8caf2e9 thru 0x8cb7ccc data_size 0x32(0x89e4 bytes) 2024-09-16T11:13:53.474 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.474 INFO:teuthology.orchestra.run.smithi162.stdout:2097 read 0x4ca39e2 thru 0x4caf856 (0xbe75 bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2098 write 0xb80b028 thru 0xb8144e2 (0x94bb bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2099 read 0xaec64a5 thru 0xaed5b1d (0xf679 bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2103 punch from 0x6edc71b to 0x6edf727, (0x300c bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2107 read 0x8d79ae7 thru 0x8d888cb (0xede5 bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2108 write 0x538413a thru 0x53847da (0x6a1 bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2109 punch from 0x9575b31 to 0x957ba1b, (0x5eea bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2111 read 0x1bb3588 thru 0x1bb50e1 (0x1b5a bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2114 read 0x7b2e52d thru 0x7b3daee (0xf5c2 bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2115 punch from 0x8125835 to 0x8127683, (0x1e4e bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2116 read 0x96d8d4f thru 0x96e5cd2 (0xcf84 bytes) 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.475 INFO:teuthology.orchestra.run.smithi162.stdout:2118 read 0x37b32e thru 0x386ccc (0xb99f bytes) 2024-09-16T11:13:53.476 INFO:teuthology.orchestra.run.smithi162.stdout:2120 write 0x9455238 thru 0x945de26 (0x8bef bytes) 2024-09-16T11:13:53.486 INFO:teuthology.orchestra.run.smithi162.stdout:2121 punch from 0xf276be to 0xf3644d, (0xed8f bytes) 2024-09-16T11:13:53.486 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.500 INFO:teuthology.orchestra.run.smithi162.stdout:2123 write 0xd80d37d thru 0xd80f833 (0x24b7 bytes) 2024-09-16T11:13:53.506 INFO:teuthology.orchestra.run.smithi162.stdout:2124 writesame 0x41c51a8 thru 0x41c594a data_size 0x11(0x7a3 bytes) 2024-09-16T11:13:53.511 INFO:teuthology.orchestra.run.smithi162.stdout:2127 writesame 0x236e5fa thru 0x23791af data_size 0x1f(0xabb6 bytes) 2024-09-16T11:13:53.540 INFO:teuthology.orchestra.run.smithi162.stdout:2128 trunc from 0xd80f834 to 0x73fe4d7 2024-09-16T11:13:53.567 INFO:teuthology.orchestra.run.smithi162.stdout:2129 trunc from 0x73fe4d7 to 0x4dcf519 2024-09-16T11:13:53.572 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:53.565+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:53.576 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:53.577 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:53.577 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:53.577 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:53.586 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:53.612 INFO:teuthology.orchestra.run.smithi162.stdout:2131 write 0xc5cfaa3 thru 0xc5d17b0 (0x1d0e bytes) 2024-09-16T11:13:53.620 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.626 INFO:teuthology.orchestra.run.smithi162.stdout:2133 trunc from 0xc5d17b1 to 0x9527c36 2024-09-16T11:13:53.649 INFO:teuthology.orchestra.run.smithi162.stdout:2134 trunc from 0x9527c36 to 0x93d21de 2024-09-16T11:13:53.661 INFO:teuthology.orchestra.run.smithi162.stdout:2135 read 0x5e757f4 thru 0x5e7e5b7 (0x8dc4 bytes) 2024-09-16T11:13:53.673 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:53.665+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:53.681 INFO:teuthology.orchestra.run.smithi162.stdout:2136 write 0xbff8c00 thru 0xc0059b9 (0xcdba bytes) 2024-09-16T11:13:53.686 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:53.686 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:53.686 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:53.686 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:53.686 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:53.696 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:53.697 INFO:teuthology.orchestra.run.smithi162.stdout:2137 write 0x64001ad thru 0x64040ae (0x3f02 bytes) 2024-09-16T11:13:53.703 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.718 INFO:teuthology.orchestra.run.smithi162.stdout:2141 trunc from 0xc0059ba to 0x54c9d6d 2024-09-16T11:13:53.740 INFO:teuthology.orchestra.run.smithi162.stdout:2143 punch from 0xcf1050 to 0xcfe6b2, (0xd662 bytes) 2024-09-16T11:13:53.740 INFO:teuthology.orchestra.run.smithi162.stdout:2144 read 0x30e19b0 thru 0x30ec901 (0xaf52 bytes) 2024-09-16T11:13:53.757 INFO:teuthology.orchestra.run.smithi162.stdout:2145 writesame 0x78f162f thru 0x78fe51d data_size 0x19(0xceef bytes) 2024-09-16T11:13:53.757 INFO:teuthology.orchestra.run.smithi162.stdout:2148 write 0x57db443 thru 0x57ddec6 (0x2a84 bytes) 2024-09-16T11:13:53.765 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.765 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.774 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:53.768+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:53.787 INFO:teuthology.orchestra.run.smithi162.stdout:2151 write 0xbf404c1 thru 0xbf4e1c3 (0xdd03 bytes) 2024-09-16T11:13:53.810 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.810 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.810 INFO:teuthology.orchestra.run.smithi162.stdout:2154 read 0xa35b456 thru 0xa3611db (0x5d86 bytes) 2024-09-16T11:13:53.810 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.810 INFO:teuthology.orchestra.run.smithi162.stdout:2157 writesame 0x64555fa thru 0x6456593 data_size 0x1(0xf9a bytes) 2024-09-16T11:13:53.817 INFO:teuthology.orchestra.run.smithi162.stdout:2158 punch from 0x4ef4b92 to 0x4efca5b, (0x7ec9 bytes) 2024-09-16T11:13:53.817 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.817 INFO:teuthology.orchestra.run.smithi162.stdout:2160 write 0xb6b5af1 thru 0xb6baacf (0x4fdf bytes) 2024-09-16T11:13:53.828 INFO:teuthology.orchestra.run.smithi162.stdout:2164 read 0x23b9aeb thru 0x23c51c5 (0xb6db bytes) 2024-09-16T11:13:53.829 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.831 INFO:teuthology.orchestra.run.smithi162.stdout:2167 trunc from 0xbf4e1c4 to 0xb1db39b 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 32.986644290999998, 2024-09-16T11:13:53.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:13:53.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8186726420000001, 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:13:53.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:13:53.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:13:53.862 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8186610810000001, 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:13:53.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:13:53.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8186478790000002, 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.867 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:13:53.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:13:53.871 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8186354390000004, 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:13:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:13:53.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8186219279999998, 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:13:53.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:13:53.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:13:53.880 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8186110129999999, 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:13:53.884 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185850949999995, 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185696409999998, 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:13:53.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:13:53.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185597549999999, 2024-09-16T11:13:53.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:13:53.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185499920000003, 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.898 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:13:53.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185398150000003, 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.902 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185233329999999, 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:13:53.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8185086359999998, 2024-09-16T11:13:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:13:53.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8184901929999997, 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.916 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:13:53.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:13:53.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8184732439999998, 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:13:53.920 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:13:53.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:13:53.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8184540309999999, 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.925 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:13:53.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8184368150000001, 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.929 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.8184155610000001, 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:13:53.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.937 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.818395593, 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:13:53.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:13:53.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:53.942 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:53.945 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:53.946 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:53.937+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2170 write 0x622e802 thru 0x623432a (0x5b29 bytes) 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2172 trunc from 0xb1db39b to 0xd114033 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2173 read 0x8c9406f thru 0x8c9b268 (0x71fa bytes) 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2174 read 0x8ec9e3e thru 0x8ecab94 (0xd57 bytes) 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2178 read 0xc5fa5cb thru 0xc606c30 (0xc666 bytes) 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2180 write 0x66ae1f7 thru 0x66b64ea (0x82f4 bytes) 2024-09-16T11:13:53.947 INFO:teuthology.orchestra.run.smithi162.stdout:2181 punch from 0x5f2245 to 0x6020ea, (0xfea5 bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2182 write 0x217e95a thru 0x21893c3 (0xaa6a bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2185 read 0x54a94b8 thru 0x54b07fd (0x7346 bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2186 read 0x5424cbc thru 0x54281e4 (0x3529 bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2187 write 0x36e476b thru 0x36ec0f4 (0x798a bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2189 read 0x9580a71 thru 0x9587454 (0x69e4 bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2190 trunc from 0xd114033 to 0xd48f20b 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2191 punch from 0x59b0426 to 0x59b5860, (0x543a bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2192 writesame 0x1b77c95 thru 0x1b7e7aa data_size 0x9(0x6b16 bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2194 punch from 0x19768f8 to 0x197a780, (0x3e88 bytes) 2024-09-16T11:13:53.948 INFO:teuthology.orchestra.run.smithi162.stdout:2195 writesame 0x590cbbb thru 0x590d6bd data_size 0x1(0xb03 bytes) 2024-09-16T11:13:53.951 INFO:teuthology.orchestra.run.smithi162.stdout:2196 write 0x2c5092e thru 0x2c54ce7 (0x43ba bytes) 2024-09-16T11:13:53.962 INFO:teuthology.orchestra.run.smithi162.stdout:2199 read 0x631ba74 thru 0x6320240 (0x47cd bytes) 2024-09-16T11:13:53.963 INFO:teuthology.orchestra.run.smithi162.stdout:2200 punch from 0xc1efb6c to 0xc1f9fea, (0xa47e bytes) 2024-09-16T11:13:53.963 INFO:teuthology.orchestra.run.smithi162.stdout:2201 write 0x7cceb71 thru 0x7cda3ec (0xb87c bytes) 2024-09-16T11:13:53.996 INFO:teuthology.orchestra.run.smithi162.stdout:2203 writesame 0xed5899a thru 0xed68865 data_size 0x2e(0xfecc bytes) 2024-09-16T11:13:54.009 INFO:teuthology.orchestra.run.smithi162.stdout:2204 trunc from 0xed68866 to 0x83fc2c1 2024-09-16T11:13:54.026 INFO:teuthology.orchestra.run.smithi162.stdout:2206 trunc from 0x83fc2c1 to 0xb719b49 2024-09-16T11:13:54.039 INFO:teuthology.orchestra.run.smithi162.stdout:2207 write 0x696b7e9 thru 0x6976e80 (0xb698 bytes) 2024-09-16T11:13:54.045 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:54.040+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:54.051 INFO:teuthology.orchestra.run.smithi162.stdout:2209 punch from 0x229da80 to 0x22acf5e, (0xf4de bytes) 2024-09-16T11:13:54.051 INFO:teuthology.orchestra.run.smithi162.stdout:2212 read 0xa5d7fe4 thru 0xa5e17ce (0x97eb bytes) 2024-09-16T11:13:54.051 INFO:teuthology.orchestra.run.smithi162.stdout:2213 read 0x17568d0 thru 0x175bf6c (0x569d bytes) 2024-09-16T11:13:54.052 INFO:teuthology.orchestra.run.smithi162.stdout:2214 read 0x12d51b thru 0x130f3e (0x3a24 bytes) 2024-09-16T11:13:54.059 INFO:teuthology.orchestra.run.smithi162.stdout:2215 trunc from 0xb719b49 to 0xe28a823 2024-09-16T11:13:54.059 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:54.059 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:54.059 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:54.059 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:54.069 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:54.084 INFO:teuthology.orchestra.run.smithi162.stdout:2216 write 0x1cb362f thru 0x1cc2de9 (0xf7bb bytes) 2024-09-16T11:13:54.100 INFO:teuthology.orchestra.run.smithi162.stdout:2218 read 0x2ed4249 thru 0x2ee18cc (0xd684 bytes) 2024-09-16T11:13:54.110 INFO:teuthology.orchestra.run.smithi162.stdout:2219 write 0xe4a44b7 thru 0xe4aace7 (0x6831 bytes) 2024-09-16T11:13:54.121 INFO:teuthology.orchestra.run.smithi162.stdout:2220 read 0x75ae115 thru 0x75bd34c (0xf238 bytes) 2024-09-16T11:13:54.121 INFO:teuthology.orchestra.run.smithi162.stdout:2221 clone 17 order 25 su 131072 sc 3 2024-09-16T11:13:54.147 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:54.141+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:54.168 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:54.168 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:54.168 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:54.169 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:54.169 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:54.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:54.248 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:54.241+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:54.277 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.087360265, 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:13:54.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:13:54.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0830017380000001, 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:13:54.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50301 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0793828559999996, 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11841462, 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50301 2024-09-16T11:13:54.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193762+0000", 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7399999999999998e-07 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193764+0000", 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193771+0000", 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-09-16T11:13:54.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193775+0000", 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3619999999999999e-06 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311756+0000", 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11798077999999999 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311868+0000", 2024-09-16T11:13:54.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112605 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312176+0000", 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030757000000000001 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.292 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0793632080000002, 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0793444220000001, 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:13:54.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0793270990000003, 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:13:54.302 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:13:54.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:13:54.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.07930963, 2024-09-16T11:13:54.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:13:54.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0754691709999999, 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:13:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:13:54.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0754590559999997, 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.317 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:54.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:13:54.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:13:54.321 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0754491750000001, 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.322 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:54.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:54.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:54.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:13:54.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:13:54.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:13:54.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:13:54.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:13:54.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:13:54.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:13:54.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:13:54.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:13:54.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:13:54.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:13:54.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:13:54.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:13:54.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.331 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.331 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.331 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.331 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0754399990000003, 2024-09-16T11:13:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:13:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.332 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.333 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.333 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:13:54.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.333 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:54.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:54.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:54.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:13:54.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:13:54.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:13:54.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:13:54.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:13:54.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:13:54.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:13:54.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:13:54.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:13:54.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:13:54.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:13:54.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:13:54.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:13:54.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.341 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.342 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.342 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:13:54.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0754239319999996, 2024-09-16T11:13:54.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:13:54.343 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.344 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.344 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:13:54.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.344 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:13:54.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:13:54.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:13:54.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:13:54.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:13:54.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:13:54.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:13:54.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:13:54.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:13:54.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:13:54.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:13:54.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.352 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.353 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.353 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:54.353 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0754075370000002, 2024-09-16T11:13:54.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:13:54.354 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.354 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:54.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.354 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:13:54.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.355 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:54.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:54.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:54.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:13:54.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:13:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:13:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:13:54.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:13:54.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:13:54.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:13:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:13:54.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:13:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:13:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:13:54.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:13:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:13:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.363 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:54.364 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0753979630000003, 2024-09-16T11:13:54.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:13:54.365 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.365 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.365 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.365 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.365 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:54.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:54.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:13:54.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:13:54.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:13:54.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:13:54.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:13:54.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:13:54.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:13:54.372 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.075388427, 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:13:54.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:13:54.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50319 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0713420329999996, 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118443188, 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50319 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:13:54.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201804+0000", 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-06 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201807+0000", 2024-09-16T11:13:54.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201809+0000", 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320222+0000", 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118412609 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320245+0000", 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3805999999999999e-05 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0713317699999996, 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:13:54.382 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:13:54.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.387 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:54.387 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0713223879999996, 2024-09-16T11:13:54.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:13:54.387 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.387 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.387 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.388 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:13:54.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.388 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:54.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:13:54.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.07131373, 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:13:54.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:13:54.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:13:54.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.0713044959999998, 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:54.400 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.406 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:54.407 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:13:54.408 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:54.409 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:54.401+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:54.494 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:54.508 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:54.500+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:54.543 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone16 from 0xe4aace8 (overlap 0xd9920c) to 0x0 2024-09-16T11:13:54.560 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:54.560 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:54.560 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:54.560 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:54.572 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:54.610 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:54.601+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:54.668 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:54.673 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:54.673 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:54.673 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:54.673 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:54.673 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:54.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:54.697 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:54.711 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:54.705+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:54.723 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:54.741 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:54.762 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:54.769 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-16T11:13:54.789 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 65.008348239, 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.620422539000003, 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.620271208999995, 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.620219723000005, 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.811 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:54.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:54.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.816 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.620149574999999, 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:54.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:54.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:54.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.619977105000004, 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:54.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:54.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.828 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.619918783000003, 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:54.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:54.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:54.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.619766663000007, 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.836 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:54.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.619588144000005, 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.842 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:54.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.847 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.619555171000002, 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:54.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:54.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.067389679000001, 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:54.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:54.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:54.859 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.067367446000006, 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:54.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:54.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:54.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:54.865 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.067238649000004, 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:54.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:54.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:54.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.067180304000004, 2024-09-16T11:13:54.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:54.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:54.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:54.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.877 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.067125915000005, 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:54.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:54.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 64.066991607999995, 2024-09-16T11:13:54.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:54.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:54.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:54.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.285894087999999, 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:54.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:54.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.285836238999998, 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:54.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:54.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.900 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.285748927, 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.901 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:54.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.285510799000001, 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:54.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:54.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:54.912 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:54.914 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:54.914 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:54.914 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:54.915 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:54.905+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:54.915 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:54.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:54.917 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:54.917 INFO:teuthology.orchestra.run.smithi162.stdout:2223 write 0x156cde5 thru 0x157bcc4 (0xeee0 bytes) 2024-09-16T11:13:54.984 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:54.984 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:54.984 INFO:teuthology.orchestra.run.smithi162.stdout:2227 read 0xa552916 thru 0xa55cdb7 (0xa4a2 bytes) 2024-09-16T11:13:54.999 INFO:teuthology.orchestra.run.smithi162.stdout:2229 punch from 0x52a12af to 0x52a3d6c, (0x2abd bytes) 2024-09-16T11:13:54.999 INFO:teuthology.orchestra.run.smithi162.stdout:2230 write 0x9b9d890 thru 0x9b9fe1b (0x258c bytes) 2024-09-16T11:13:55.014 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:55.009+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:55.035 INFO:teuthology.orchestra.run.smithi162.stdout:2233 trunc from 0xe4aace8 to 0x594555e 2024-09-16T11:13:55.079 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.079 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.094 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:55.117 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:55.108+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:55.216 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:55.209+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:55.221 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.221 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.227 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.228 INFO:teuthology.orchestra.run.smithi162.stdout:2235 write 0x3704964 thru 0x3706a96 (0x2133 bytes) 2024-09-16T11:13:55.230 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:55.231 INFO:teuthology.orchestra.run.smithi162.stdout:2237 read 0x44869c3 thru 0x44944b9 (0xdaf7 bytes) 2024-09-16T11:13:55.246 INFO:teuthology.orchestra.run.smithi162.stdout:2238 write 0x84e4a95 thru 0x84e5223 (0x78f bytes) 2024-09-16T11:13:55.259 INFO:teuthology.orchestra.run.smithi162.stdout:2239 trunc from 0x84e5224 to 0x467917f 2024-09-16T11:13:55.287 INFO:teuthology.orchestra.run.smithi162.stdout:2241 read 0x49c7f0 thru 0x4a5ed9 (0x96ea bytes) 2024-09-16T11:13:55.311 INFO:teuthology.orchestra.run.smithi162.stdout:2243 write 0xce82b41 thru 0xce8d5ba (0xaa7a bytes) 2024-09-16T11:13:55.317 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:55.309+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:55.318 INFO:teuthology.orchestra.run.smithi162.stdout:2244 write 0xb9ffa32 thru 0xba0e6af (0xec7e bytes) 2024-09-16T11:13:55.328 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.329 INFO:teuthology.orchestra.run.smithi162.stdout:2248 write 0x16c18a9 thru 0x16c8132 (0x688a bytes) 2024-09-16T11:13:55.334 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.334 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.342 INFO:teuthology.orchestra.run.smithi162.stdout:2249 write 0xe3dfb1f thru 0xe3e949b (0x997d bytes) 2024-09-16T11:13:55.344 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:55.354 INFO:teuthology.orchestra.run.smithi162.stdout:2250 trunc from 0xe3e949c to 0xbcf7f72 2024-09-16T11:13:55.376 INFO:teuthology.orchestra.run.smithi162.stdout:2252 read 0x645f3e1 thru 0x6465ec4 (0x6ae4 bytes) 2024-09-16T11:13:55.377 INFO:teuthology.orchestra.run.smithi162.stdout:2253 read 0xa14df93 thru 0xa150865 (0x28d3 bytes) 2024-09-16T11:13:55.377 INFO:teuthology.orchestra.run.smithi162.stdout:2254 read 0xadfcfbd thru 0xae05b77 (0x8bbb bytes) 2024-09-16T11:13:55.378 INFO:teuthology.orchestra.run.smithi162.stdout:2255 punch from 0xa98331 to 0xa9c3dd, (0x40ac bytes) 2024-09-16T11:13:55.378 INFO:teuthology.orchestra.run.smithi162.stdout:2256 write 0xa6ba7b6 thru 0xa6c5a1d (0xb268 bytes) 2024-09-16T11:13:55.384 INFO:teuthology.orchestra.run.smithi162.stdout:2257 read 0x1131d40 thru 0x113fa39 (0xdcfa bytes) 2024-09-16T11:13:55.385 INFO:teuthology.orchestra.run.smithi162.stdout:2260 read 0xac83a66 thru 0xac845b3 (0xb4e bytes) 2024-09-16T11:13:55.386 INFO:teuthology.orchestra.run.smithi162.stdout:2261 punch from 0x848e42f to 0x8494196, (0x5d67 bytes) 2024-09-16T11:13:55.395 INFO:teuthology.orchestra.run.smithi162.stdout:2262 trunc from 0xbcf7f72 to 0x74a8b44 2024-09-16T11:13:55.412 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:13:55.414 INFO:teuthology.orchestra.run.smithi162.stdout:2263 read 0x1700ae9 thru 0x170dee8 (0xd400 bytes) 2024-09-16T11:13:55.418 INFO:teuthology.orchestra.run.smithi162.stdout:2264 punch from 0x29a1cc1 to 0x29a3828, (0x1b67 bytes) 2024-09-16T11:13:55.418 INFO:teuthology.orchestra.run.smithi162.stdout:2265 write 0x61e9036 thru 0x61f2d23 (0x9cee bytes) 2024-09-16T11:13:55.418 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:55.412+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:55.420 INFO:teuthology.orchestra.run.smithi162.stdout:2266 read 0x59726f2 thru 0x597ea4d (0xc35c bytes) 2024-09-16T11:13:55.421 INFO:teuthology.orchestra.run.smithi162.stdout:2268 writesame 0x1e7e1c6 thru 0x1e8cbaf data_size 0x2(0xe9ea bytes) 2024-09-16T11:13:55.444 INFO:teuthology.orchestra.run.smithi162.stdout:2271 trunc from 0x74a8b44 to 0x61872bb 2024-09-16T11:13:55.464 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.464 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.472 INFO:teuthology.orchestra.run.smithi162.stdout:2272 write 0x778d7ab thru 0x7794011 (0x6867 bytes) 2024-09-16T11:13:55.474 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:55.475 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.481 INFO:teuthology.orchestra.run.smithi162.stdout:2274 trunc from 0x7794012 to 0x3fd39a3 2024-09-16T11:13:55.504 INFO:teuthology.orchestra.run.smithi162.stdout:2275 punch from 0x32b8525 to 0x32c1659, (0x9134 bytes) 2024-09-16T11:13:55.504 INFO:teuthology.orchestra.run.smithi162.stdout:2277 punch from 0x1739a57 to 0x1746721, (0xccca bytes) 2024-09-16T11:13:55.519 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:55.512+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:55.528 INFO:teuthology.orchestra.run.smithi162.stdout:2279 write 0xcb3cc72 thru 0xcb432bf (0x664e bytes) 2024-09-16T11:13:55.541 INFO:teuthology.orchestra.run.smithi162.stdout:2280 write 0x1c7a72 thru 0x1d7623 (0xfbb2 bytes) 2024-09-16T11:13:55.545 INFO:teuthology.orchestra.run.smithi162.stdout:2282 read 0x75ba93a thru 0x75c4fea (0xa6b1 bytes) 2024-09-16T11:13:55.545 INFO:teuthology.orchestra.run.smithi162.stdout:2284 write 0x14e6a49 thru 0x14ec4ee (0x5aa6 bytes) 2024-09-16T11:13:55.549 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.549 INFO:teuthology.orchestra.run.smithi162.stdout:2286 read 0xbefcdff thru 0xbf04bb9 (0x7dbb bytes) 2024-09-16T11:13:55.549 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.549 INFO:teuthology.orchestra.run.smithi162.stdout:2288 punch from 0xb5d91df to 0xb5e8f9f, (0xfdc0 bytes) 2024-09-16T11:13:55.549 INFO:teuthology.orchestra.run.smithi162.stdout:2289 punch from 0x47cdbb7 to 0x47da972, (0xcdbb bytes) 2024-09-16T11:13:55.565 INFO:teuthology.orchestra.run.smithi162.stdout:2290 trunc from 0xcb432c0 to 0x30a6e4b 2024-09-16T11:13:55.573 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.573 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:55.590 INFO:teuthology.orchestra.run.smithi162.stdout:2291 punch from 0x29d7f12 to 0x29db3ef, (0x34dd bytes) 2024-09-16T11:13:55.590 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.602 INFO:teuthology.orchestra.run.smithi162.stdout:2294 trunc from 0x30a6e4b to 0xa7642ff 2024-09-16T11:13:55.613 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.613 INFO:teuthology.orchestra.run.smithi162.stdout:2299 writesame 0x3e31afe thru 0x3e3654f data_size 0x2a(0x4a52 bytes) 2024-09-16T11:13:55.620 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:55.612+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:55.634 INFO:teuthology.orchestra.run.smithi162.stdout:2301 writesame 0xecbbbcd thru 0xecc8955 data_size 0x1(0xcd89 bytes) 2024-09-16T11:13:55.634 INFO:teuthology.orchestra.run.smithi162.stdout:2302 write 0xd11ece thru 0xd1503c (0x316f bytes) 2024-09-16T11:13:55.637 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:55.637 INFO:teuthology.orchestra.run.smithi162.stdout:2305 write 0x9228408 thru 0x922e99b (0x6594 bytes) 2024-09-16T11:13:55.642 INFO:teuthology.orchestra.run.smithi162.stdout:2306 clone 18 order 25 su 1048576 sc 3 2024-09-16T11:13:55.687 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.687 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.695 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:55.721 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:55.713+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:55.822 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:55.813+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:55.837 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.838 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.846 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:55.923 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:55.917+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:55.941 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:55.941 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:55.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:56.024 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:56.017+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:56.051 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.051 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.060 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:56.125 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:56.120+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:56.163 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.164 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:56.226 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:56.220+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:56.316 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.316 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.325 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:56.327 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:56.321+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:56.396 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:13:56.396 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:13:56.396 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T11:13:56.396 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:13:56.427 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:56.421+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:56.431 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.431 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.441 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:56.528 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:56.521+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:56.553 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.553 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.563 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:56.585 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone17 from 0xecc8956 (overlap 0x30a6e4b) to 0x0 2024-09-16T11:13:56.629 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:56.624+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:56.658 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.658 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.667 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:13:56.706 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:13:56.706 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:56.706 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 64, 2024-09-16T11:13:56.706 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7e3ff0c6-8090-4b11-95d6-fd611813de49", 2024-09-16T11:13:56.706 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2024-09-16T11:12:35.459871+0000", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2024-09-16T11:13:56.406206+0000", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2024-09-16T11:12:46.645944+0000", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2024-09-16T11:13:18.117664+0000", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5804032, 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "nodeep-scrub", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2024-09-16T11:13:56.707 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 4, 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "quincy", 2024-09-16T11:13:56.708 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": ".mgr", 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:48.004741+0000", 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:56.709 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2024-09-16T11:13:56.710 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:56.711 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:56.711 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:56.711 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:56.711 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:56.711 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:56.711 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "17", 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2024-09-16T11:13:56.712 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:56.714 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_max": 32, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:56.715 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": {} 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:53.361644+0000", 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:56.716 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:56.717 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "26", 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:13:56.718 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [ 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: 3 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:56.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": 3, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": 3, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:56.720 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:56.721 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:56.721 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:56.721 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:56.721 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:56.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:56.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:56.722 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 3, 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cache", 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:02.524586+0000", 2024-09-16T11:13:56.723 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8201, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:56.724 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 4, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 4, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 4, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 4, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 4, 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:56.725 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "30", 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:13:56.726 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": 2, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "writeback", 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 250, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:56.727 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "bloom", 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "target_size": 0, 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "seed": 0 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.728 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 60, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 8, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [ 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.729 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:56.730 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "pool_client.0", 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:11.876887+0000", 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:13:56.731 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:13:56.732 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:13:56.733 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:13:56.733 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 32, 2024-09-16T11:13:56.733 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 32, 2024-09-16T11:13:56.733 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 32, 2024-09-16T11:13:56.733 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 32, 2024-09-16T11:13:56.733 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 32, 2024-09-16T11:13:56.734 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:13:56.734 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:13:56.734 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:13:56.734 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:13:56.734 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "64", 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:13:56.735 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 21, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 64, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:13:56.736 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:13:56.737 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:13:56.738 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:13:56.739 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b772f030-6c52-46c6-a992-36cadd7f9edd", 2024-09-16T11:13:56.740 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.740 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.740 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.740 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.740 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.740 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.741 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.741 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 60, 2024-09-16T11:13:56.741 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.741 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.741 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.741 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.742 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2024-09-16T11:13:56.742 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.744 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.744 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2024-09-16T11:13:56.744 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2024-09-16T11:13:56.745 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2024-09-16T11:13:56.746 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6825/22248", 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6827/22248", 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6831/22248", 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6829/22248", 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.747 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6335aa8f-a82b-45a4-be04-b1b064314f8d", 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 0.364288330078125, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.748 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6800", 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6801", 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.752 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6801/22247", 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6803/22247", 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6807/22247", 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6805/22247", 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.754 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "8d9432da-1713-44e1-9c96-db33f21cf333", 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 0, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 0, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.755 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 31, 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.756 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.757 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.758 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6817/22258", 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6819/22258", 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6823/22258", 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6821/22258", 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.760 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "f17d5292-6b43-4887-a146-a647c60e4f86", 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.761 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 60, 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.764 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2024-09-16T11:13:56.765 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6809/22257", 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6811/22257", 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6815/22257", 2024-09-16T11:13:56.766 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6813/22257", 2024-09-16T11:13:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "93cb4050-ba1d-44c4-8e05-854a4cbb9236", 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.768 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6800", 2024-09-16T11:13:56.769 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6801", 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6802", 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6803", 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.771 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6806", 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6807", 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6804", 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6805", 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:13:56.773 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6801/22379", 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6803/22379", 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6807/22379", 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6805/22379", 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.774 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "432a0446-4f3b-436d-bba0-76184260e13c", 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.775 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.776 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 36, 2024-09-16T11:13:56.776 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.776 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.776 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.776 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.777 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6808", 2024-09-16T11:13:56.777 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6809", 2024-09-16T11:13:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6810", 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.780 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6811", 2024-09-16T11:13:56.780 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.780 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6814", 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6815", 2024-09-16T11:13:56.781 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6812", 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6813", 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:13:56.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6809/22380", 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6811/22380", 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6815/22380", 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6813/22380", 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "7c7e65e2-ab06-47f4-8d6d-712aa308e13f", 2024-09-16T11:13:56.783 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 60, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.784 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6816", 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6818", 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.786 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6819", 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6821", 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6826", 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6827", 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6822", 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6824", 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:13:56.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6818/22381", 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6821/22381", 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6827/22381", 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6824/22381", 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.790 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "827daf60-a01e-40a5-9595-7ec5406e4a8a", 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 18, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:13:56.791 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6817", 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6820", 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6823", 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6825", 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6830", 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6831", 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6828", 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6829", 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:13:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6820/22378", 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6825/22378", 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6831/22378", 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6829/22378", 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2024-09-16T11:13:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.054827+0000", 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.474331+0000", 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 65536, 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.753279+0000", 2024-09-16T11:13:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.733978+0000", 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:13:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.980000+0000", 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:13:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.142568+0000", 2024-09-16T11:13:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:13:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.883591+0000", 2024-09-16T11:13:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.129808+0000", 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklist": {}, 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "range_blocklist": {}, 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2024-09-16T11:13:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [], 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [], 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {}, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode": { 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_enabled": false, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_bucket_count": 0, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:13:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_bucket": 0 2024-09-16T11:13:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:56.807 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:56.809 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.809 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.810 INFO:tasks.thrashosds.thrasher:j is {'epoch': 64, 'fsid': '7e3ff0c6-8090-4b11-95d6-fd611813de49', 'created': '2024-09-16T11:12:35.459871+0000', 'modified': '2024-09-16T11:13:56.406206+0000', 'last_up_change': '2024-09-16T11:12:46.645944+0000', 'last_in_change': '2024-09-16T11:13:18.117664+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:12:48.004741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:12:53.361644+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:13:02.524586+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:13:11.876887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'b772f030-6c52-46c6-a992-36cadd7f9edd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6825', 'nonce': 22248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6826', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 22248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6830', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 22248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6829', 'nonce': 22248}]}, 'public_addr': '172.21.15.162:6825/22248', 'cluster_addr': '172.21.15.162:6827/22248', 'heartbeat_back_addr': '172.21.15.162:6831/22248', 'heartbeat_front_addr': '172.21.15.162:6829/22248', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6335aa8f-a82b-45a4-be04-b1b064314f8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.364288330078125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 22247}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 22247}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 22247}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 22247}]}, 'public_addr': '172.21.15.162:6801/22247', 'cluster_addr': '172.21.15.162:6803/22247', 'heartbeat_back_addr': '172.21.15.162:6807/22247', 'heartbeat_front_addr': '172.21.15.162:6805/22247', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8d9432da-1713-44e1-9c96-db33f21cf333', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6817', 'nonce': 22258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6818', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 22258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6822', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 22258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6821', 'nonce': 22258}]}, 'public_addr': '172.21.15.162:6817/22258', 'cluster_addr': '172.21.15.162:6819/22258', 'heartbeat_back_addr': '172.21.15.162:6823/22258', 'heartbeat_front_addr': '172.21.15.162:6821/22258', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f17d5292-6b43-4887-a146-a647c60e4f86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 22257}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 22257}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 22257}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 22257}]}, 'public_addr': '172.21.15.162:6809/22257', 'cluster_addr': '172.21.15.162:6811/22257', 'heartbeat_back_addr': '172.21.15.162:6815/22257', 'heartbeat_front_addr': '172.21.15.162:6813/22257', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '93cb4050-ba1d-44c4-8e05-854a4cbb9236', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 22379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 22379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 22379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 22379}]}, 'public_addr': '172.21.15.165:6801/22379', 'cluster_addr': '172.21.15.165:6803/22379', 'heartbeat_back_addr': '172.21.15.165:6807/22379', 'heartbeat_front_addr': '172.21.15.165:6805/22379', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '432a0446-4f3b-436d-bba0-76184260e13c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 22380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 22380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 22380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 22380}]}, 'public_addr': '172.21.15.165:6809/22380', 'cluster_addr': '172.21.15.165:6811/22380', 'heartbeat_back_addr': '172.21.15.165:6815/22380', 'heartbeat_front_addr': '172.21.15.165:6813/22380', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c7e65e2-ab06-47f4-8d6d-712aa308e13f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6818', 'nonce': 22381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6819', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 22381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 22381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6824', 'nonce': 22381}]}, 'public_addr': '172.21.15.165:6818/22381', 'cluster_addr': '172.21.15.165:6821/22381', 'heartbeat_back_addr': '172.21.15.165:6827/22381', 'heartbeat_front_addr': '172.21.15.165:6824/22381', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '827daf60-a01e-40a5-9595-7ec5406e4a8a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6817', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6820', 'nonce': 22378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6823', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 22378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 22378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 22378}]}, 'public_addr': '172.21.15.165:6820/22378', 'cluster_addr': '172.21.15.165:6825/22378', 'heartbeat_back_addr': '172.21.15.165:6831/22378', 'heartbeat_front_addr': '172.21.15.165:6829/22378', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.054827+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.474331+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.753279+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.733978+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.980000+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.142568+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.883591+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.129808+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:13:56.810 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T11:13:56.811 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:13:56.811 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:56.801+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:56.811 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-16T11:13:56.812 INFO:teuthology.orchestra.run.smithi162.stdout:2307 read 0x3a1994c thru 0x3a2905e (0xf713 bytes) 2024-09-16T11:13:56.812 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:56.812 INFO:teuthology.orchestra.run.smithi162.stdout:2309 read 0x2f8cff0 thru 0x2f8dea3 (0xeb4 bytes) 2024-09-16T11:13:56.812 INFO:teuthology.orchestra.run.smithi162.stdout:2310 trunc from 0xecc8956 to 0x79d9aba 2024-09-16T11:13:56.899 INFO:teuthology.orchestra.run.smithi162.stdout:2311 read 0x69ff2f4 thru 0x6a0c06a (0xcd77 bytes) 2024-09-16T11:13:56.906 INFO:teuthology.orchestra.run.smithi162.stdout:2313 trunc from 0x79d9aba to 0xa98ea65 2024-09-16T11:13:56.909 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:56.901+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:56.917 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:13:56.918 INFO:teuthology.orchestra.run.smithi162.stdout:2314 read 0xa632685 thru 0xa636765 (0x40e1 bytes) 2024-09-16T11:13:56.932 INFO:teuthology.orchestra.run.smithi162.stdout:2315 write 0xb5bd52e thru 0xb5c519c (0x7c6f bytes) 2024-09-16T11:13:56.935 INFO:teuthology.orchestra.run.smithi162.stdout:2316 writesame 0x219e75a thru 0x21a6bd1 data_size 0x36(0x8478 bytes) 2024-09-16T11:13:56.954 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:56.955 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:56.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:13:57.008 INFO:teuthology.orchestra.run.smithi162.stdout:2317 punch from 0x37b29 to 0x3d772, (0x5c49 bytes) 2024-09-16T11:13:57.008 INFO:teuthology.orchestra.run.smithi162.stdout:2318 read 0xd96899 thru 0xd994b1 (0x2c19 bytes) 2024-09-16T11:13:57.009 INFO:teuthology.orchestra.run.smithi162.stdout:2319 write 0x938d524 thru 0x93918af (0x438c bytes) 2024-09-16T11:13:57.009 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:57.001+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:57.012 INFO:teuthology.orchestra.run.smithi162.stdout:2320 read 0x7d53d21 thru 0x7d5f379 (0xb659 bytes) 2024-09-16T11:13:57.012 INFO:teuthology.orchestra.run.smithi162.stdout:2321 read 0xb30336b thru 0xb30881e (0x54b4 bytes) 2024-09-16T11:13:57.013 INFO:teuthology.orchestra.run.smithi162.stdout:2322 write 0x7de4f71 thru 0x7df0c74 (0xbd04 bytes) 2024-09-16T11:13:57.027 INFO:teuthology.orchestra.run.smithi162.stdout:2323 trunc from 0xb5c519d to 0x5883196 2024-09-16T11:13:57.048 INFO:teuthology.orchestra.run.smithi162.stdout:2324 read 0x3cbb5c2 thru 0x3cbfedc (0x491b bytes) 2024-09-16T11:13:57.049 INFO:teuthology.orchestra.run.smithi162.stdout:2325 read 0x2667ea2 thru 0x266e2b7 (0x6416 bytes) 2024-09-16T11:13:57.049 INFO:teuthology.orchestra.run.smithi162.stdout:2326 read 0x12d094d thru 0x12d797e (0x7032 bytes) 2024-09-16T11:13:57.061 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:57.061 INFO:teuthology.orchestra.run.smithi162.stdout:2327 write 0x6df1efd thru 0x6dfeb8e (0xcc92 bytes) 2024-09-16T11:13:57.064 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:57.064 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:57.067 INFO:teuthology.orchestra.run.smithi162.stdout:2328 read 0x24e7075 thru 0x24f515c (0xe0e8 bytes) 2024-09-16T11:13:57.073 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:13:57.078 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:57.087 INFO:teuthology.orchestra.run.smithi162.stdout:2329 writesame 0xb1bbbd7 thru 0xb1c8a2d data_size 0x1(0xce57 bytes) 2024-09-16T11:13:57.087 INFO:teuthology.orchestra.run.smithi162.stdout:2330 punch from 0x463816a to 0x463a068, (0x1efe bytes) 2024-09-16T11:13:57.087 INFO:teuthology.orchestra.run.smithi162.stdout:2331 punch from 0x4cded20 to 0x4ce4ae7, (0x5dc7 bytes) 2024-09-16T11:13:57.087 INFO:teuthology.orchestra.run.smithi162.stdout:2332 punch from 0xe0a5ee to 0xe0fcfc, (0x570e bytes) 2024-09-16T11:13:57.098 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:57.104 INFO:teuthology.orchestra.run.smithi162.stdout:2333 writesame 0xd8ede1b thru 0xd8f1e65 data_size 0x1(0x404b bytes) 2024-09-16T11:13:57.104 INFO:teuthology.orchestra.run.smithi162.stdout:2334 write 0x3cc173a thru 0x3cd149f (0xfd66 bytes) 2024-09-16T11:13:57.108 INFO:teuthology.orchestra.run.smithi162.stdout:2335 read 0x5923325 thru 0x592f5a4 (0xc280 bytes) 2024-09-16T11:13:57.109 INFO:teuthology.orchestra.run.smithi162.stdout:2336 punch from 0xa0cead1 to 0xa0cfaa6, (0xfd5 bytes) 2024-09-16T11:13:57.109 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.109 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:57.104+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:57.119 INFO:teuthology.orchestra.run.smithi162.stdout:2338 trunc from 0xd8f1e66 to 0x8259a55 2024-09-16T11:13:57.134 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.134 INFO:teuthology.orchestra.run.smithi162.stdout:2340 punch from 0x2b2ec92 to 0x2b32c03, (0x3f71 bytes) 2024-09-16T11:13:57.134 INFO:teuthology.orchestra.run.smithi162.stdout:2341 write 0x515ec74 thru 0x51620b2 (0x343f bytes) 2024-09-16T11:13:57.161 INFO:teuthology.orchestra.run.smithi162.stdout:2344 writesame 0xcecfd72 thru 0xced7e89 data_size 0x44(0x8118 bytes) 2024-09-16T11:13:57.161 INFO:teuthology.orchestra.run.smithi162.stdout:2345 punch from 0xabba50d to 0xabc4a25, (0xa518 bytes) 2024-09-16T11:13:57.161 INFO:teuthology.orchestra.run.smithi162.stdout:2346 read 0x908b30d thru 0x90999c0 (0xe6b4 bytes) 2024-09-16T11:13:57.161 INFO:teuthology.orchestra.run.smithi162.stdout:2348 write 0xc116ab4 thru 0xc119249 (0x2796 bytes) 2024-09-16T11:13:57.168 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.175 INFO:teuthology.orchestra.run.smithi162.stdout:2350 trunc from 0xced7e8a to 0xa12228b 2024-09-16T11:13:57.200 INFO:teuthology.orchestra.run.smithi162.stdout:2351 writesame 0x8962e74 thru 0x896a820 data_size 0x3(0x79ad bytes) 2024-09-16T11:13:57.210 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:57.201+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:57.215 INFO:teuthology.orchestra.run.smithi162.stdout:2352 writesame 0x5c68971 thru 0x5c77f85 data_size 0x3(0xf615 bytes) 2024-09-16T11:13:57.215 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:57.215 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:57.224 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:13:57.235 INFO:teuthology.orchestra.run.smithi162.stdout:2353 read 0x7987b7 thru 0x7a4d11 (0xc55b bytes) 2024-09-16T11:13:57.236 INFO:teuthology.orchestra.run.smithi162.stdout:2354 read 0x7bcbbc6 thru 0x7bcdc91 (0x20cc bytes) 2024-09-16T11:13:57.236 INFO:teuthology.orchestra.run.smithi162.stdout:2356 write 0x5d67508 thru 0x5d70e94 (0x998d bytes) 2024-09-16T11:13:57.240 INFO:teuthology.orchestra.run.smithi162.stdout:2357 read 0x366ba89 thru 0x3672cdf (0x7257 bytes) 2024-09-16T11:13:57.253 INFO:teuthology.orchestra.run.smithi162.stdout:2358 write 0xa71f5b2 thru 0xa723ea6 (0x48f5 bytes) 2024-09-16T11:13:57.256 INFO:teuthology.orchestra.run.smithi162.stdout:2359 write 0x16e3bd5 thru 0x16e9c15 (0x6041 bytes) 2024-09-16T11:13:57.259 INFO:teuthology.orchestra.run.smithi162.stdout:2360 punch from 0x36bde5 to 0x36ee79, (0x3094 bytes) 2024-09-16T11:13:57.268 INFO:teuthology.orchestra.run.smithi162.stdout:2362 trunc from 0xa723ea7 to 0xd80adc9 2024-09-16T11:13:57.279 INFO:teuthology.orchestra.run.smithi162.stdout:2363 writesame 0x949cb36 thru 0x94a1a3f data_size 0x2(0x4f0a bytes) 2024-09-16T11:13:57.306 INFO:teuthology.orchestra.run.smithi162.stdout:2364 trunc from 0xd80adc9 to 0x7bd6e48 2024-09-16T11:13:57.311 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:57.305+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:57.317 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:57.317 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:57.320 INFO:teuthology.orchestra.run.smithi162.stdout:2365 trunc from 0x7bd6e48 to 0x7dffc6d 2024-09-16T11:13:57.329 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:57.331 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:13:57.340 INFO:teuthology.orchestra.run.smithi162.stdout:2366 trunc from 0x7dffc6d to 0xc6bf85d 2024-09-16T11:13:57.354 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:57.365 INFO:teuthology.orchestra.run.smithi162.stdout:2367 write 0xd9c0682 thru 0xd9cbf75 (0xb8f4 bytes) 2024-09-16T11:13:57.371 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.371 INFO:teuthology.orchestra.run.smithi162.stdout:2371 write 0x7f80bac thru 0x7f8e759 (0xdbae bytes) 2024-09-16T11:13:57.374 INFO:teuthology.orchestra.run.smithi162.stdout:2373 writesame 0x5ee024f thru 0x5ee3926 data_size 0x28(0x36d8 bytes) 2024-09-16T11:13:57.376 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:57.377 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.389 INFO:teuthology.orchestra.run.smithi162.stdout:2375 write 0xe2606a6 thru 0xe269f4b (0x98a6 bytes) 2024-09-16T11:13:57.392 INFO:teuthology.orchestra.run.smithi162.stdout:2376 read 0xd447a15 thru 0xd44a1a2 (0x278e bytes) 2024-09-16T11:13:57.397 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:57.405 INFO:teuthology.orchestra.run.smithi162.stdout:2377 writesame 0xedf3182 thru 0xee02647 data_size 0x26(0xf4c6 bytes) 2024-09-16T11:13:57.408 INFO:teuthology.orchestra.run.smithi162.stdout:2378 read 0x47902f9 thru 0x4799573 (0x927b bytes) 2024-09-16T11:13:57.409 INFO:teuthology.orchestra.run.smithi162.stdout:2379 read 0xaf5319b thru 0xaf5f456 (0xc2bc bytes) 2024-09-16T11:13:57.411 INFO:teuthology.orchestra.run.smithi162.stdout:2381 trunc from 0xee02648 to 0xe7c9d28 2024-09-16T11:13:57.411 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:57.405+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:57.420 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:57.422 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:57.423 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:57.427 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:13:57.428 INFO:teuthology.orchestra.run.smithi162.stdout:2384 writesame 0x59689f5 thru 0x5974d3a data_size 0x5(0xc346 bytes) 2024-09-16T11:13:57.440 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.440 INFO:teuthology.orchestra.run.smithi162.stdout:2386 read 0x2cac37 thru 0x2d9b1a (0xeee4 bytes) 2024-09-16T11:13:57.441 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.441 INFO:teuthology.orchestra.run.smithi162.stdout:2389 writesame 0x6fc92f9 thru 0x6fd2c50 data_size 0x1c(0x9958 bytes) 2024-09-16T11:13:57.446 INFO:teuthology.orchestra.run.smithi162.stdout:2391 punch from 0xa8877ce to 0xa88c85e, (0x5090 bytes) 2024-09-16T11:13:57.447 INFO:teuthology.orchestra.run.smithi162.stdout:2393 writesame 0xd815fb3 thru 0xd81a9d3 data_size 0x1(0x4a21 bytes) 2024-09-16T11:13:57.469 INFO:teuthology.orchestra.run.smithi162.stdout:2394 writesame 0xcd52444 thru 0xcd5a76e data_size 0x15(0x832b bytes) 2024-09-16T11:13:57.472 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.472 INFO:teuthology.orchestra.run.smithi162.stdout:2397 write 0x815da7 thru 0x8167fb (0xa55 bytes) 2024-09-16T11:13:57.475 INFO:teuthology.orchestra.run.smithi162.stdout:2398 write 0x3a2b1 thru 0x41d2a (0x7a7a bytes) 2024-09-16T11:13:57.478 INFO:teuthology.orchestra.run.smithi162.stdout:2399 write 0xe03f468 thru 0xe04196c (0x2505 bytes) 2024-09-16T11:13:57.484 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:57.484 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:57.484 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:57.484 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:57.484 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:57.485 INFO:teuthology.orchestra.run.smithi162.stdout:2400 punch from 0x391708c to 0x3920bf2, (0x9b66 bytes) 2024-09-16T11:13:57.485 INFO:teuthology.orchestra.run.smithi162.stdout:2402 read 0x3fd1da0 thru 0x3fd8df1 (0x7052 bytes) 2024-09-16T11:13:57.485 INFO:teuthology.orchestra.run.smithi162.stdout:2403 write 0x9899217 thru 0x989cf50 (0x3d3a bytes) 2024-09-16T11:13:57.488 INFO:teuthology.orchestra.run.smithi162.stdout:2405 punch from 0x3dc5798 to 0x3dc7c09, (0x2471 bytes) 2024-09-16T11:13:57.489 INFO:teuthology.orchestra.run.smithi162.stdout:2406 read 0xd17681c thru 0xd17c8d2 (0x60b7 bytes) 2024-09-16T11:13:57.489 INFO:teuthology.orchestra.run.smithi162.stdout:2407 read 0x1035f1d thru 0x1043cb1 (0xdd95 bytes) 2024-09-16T11:13:57.490 INFO:teuthology.orchestra.run.smithi162.stdout:2408 read 0x38ea863 thru 0x38ea963 (0x101 bytes) 2024-09-16T11:13:57.491 INFO:teuthology.orchestra.run.smithi162.stdout:2409 punch from 0x7b305d to 0x7b360f, (0x5b2 bytes) 2024-09-16T11:13:57.491 INFO:teuthology.orchestra.run.smithi162.stdout:2410 read 0xab9a8ae thru 0xaba1af4 (0x7247 bytes) 2024-09-16T11:13:57.491 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.493 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:13:57.505 INFO:teuthology.orchestra.run.smithi162.stdout:2412 trunc from 0xe7c9d28 to 0x74b6e6b 2024-09-16T11:13:57.512 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:57.504+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:57.527 INFO:teuthology.orchestra.run.smithi162.stdout:2414 read 0x20407fe thru 0x204f519 (0xed1c bytes) 2024-09-16T11:13:57.528 INFO:teuthology.orchestra.run.smithi162.stdout:2415 punch from 0xccd9d5 to 0xcd53b8, (0x79e3 bytes) 2024-09-16T11:13:57.528 INFO:teuthology.orchestra.run.smithi162.stdout:2416 read 0x3a67b5f thru 0x3a74038 (0xc4da bytes) 2024-09-16T11:13:57.529 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.540 INFO:teuthology.orchestra.run.smithi162.stdout:2418 writesame 0x83d246e thru 0x83e1ced data_size 0x8e(0xf880 bytes) 2024-09-16T11:13:57.542 INFO:teuthology.orchestra.run.smithi162.stdout:2419 read 0x6c813d5 thru 0x6c90a7f (0xf6ab bytes) 2024-09-16T11:13:57.543 INFO:teuthology.orchestra.run.smithi162.stdout:2420 punch from 0x74baa7a to 0x74c5a89, (0xb00f bytes) 2024-09-16T11:13:57.543 INFO:teuthology.orchestra.run.smithi162.stdout:2423 punch from 0x67f35b4 to 0x67fb578, (0x7fc4 bytes) 2024-09-16T11:13:57.543 INFO:teuthology.orchestra.run.smithi162.stdout:2424 read 0x7688f8f thru 0x768a8f3 (0x1965 bytes) 2024-09-16T11:13:57.544 INFO:teuthology.orchestra.run.smithi162.stdout:2425 writesame 0x7137f01 thru 0x7138a59 data_size 0x7(0xb59 bytes) 2024-09-16T11:13:57.546 INFO:teuthology.orchestra.run.smithi162.stdout:2427 writesame 0x71c800e thru 0x71cfc99 data_size 0x6(0x7c8c bytes) 2024-09-16T11:13:57.551 INFO:teuthology.orchestra.run.smithi162.stdout:2428 punch from 0x702cfa9 to 0x702fe26, (0x2e7d bytes) 2024-09-16T11:13:57.565 INFO:teuthology.orchestra.run.smithi162.stdout:2429 writesame 0xa13f8a0 thru 0xa143da2 data_size 0x27(0x4503 bytes) 2024-09-16T11:13:57.567 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.567 INFO:teuthology.orchestra.run.smithi162.stdout:2431 writesame 0x24ecd34 thru 0x24fac5f data_size 0x17(0xdf2c bytes) 2024-09-16T11:13:57.571 INFO:teuthology.orchestra.run.smithi162.stdout:2433 write 0x5772d45 thru 0x577e568 (0xb824 bytes) 2024-09-16T11:13:57.573 INFO:teuthology.orchestra.run.smithi162.stdout:2436 read 0x2aad5e2 thru 0x2ab1b2c (0x454b bytes) 2024-09-16T11:13:57.573 INFO:teuthology.orchestra.run.smithi162.stdout:2438 read 0xa0f17eb thru 0xa0ffb64 (0xe37a bytes) 2024-09-16T11:13:57.575 INFO:teuthology.orchestra.run.smithi162.stdout:2440 punch from 0x8e2232e to 0x8e26f4c, (0x4c1e bytes) 2024-09-16T11:13:57.575 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.575 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.575 INFO:teuthology.orchestra.run.smithi162.stdout:2443 write 0x89521de thru 0x89529d6 (0x7f9 bytes) 2024-09-16T11:13:57.577 INFO:teuthology.orchestra.run.smithi162.stdout:2444 punch from 0x417f664 to 0x4184b45, (0x54e1 bytes) 2024-09-16T11:13:57.577 INFO:teuthology.orchestra.run.smithi162.stdout:2445 read 0x8e22ede thru 0x8e31ee3 (0xf006 bytes) 2024-09-16T11:13:57.578 INFO:teuthology.orchestra.run.smithi162.stdout:2447 writesame 0x38a094d thru 0x38a3cd8 data_size 0x2(0x338c bytes) 2024-09-16T11:13:57.593 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:57.593 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.4036839850000007, 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:13:57.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.598 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3993254579999999, 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:13:57.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:13:57.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50301 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3957065760000003, 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11841462, 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50301 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193762+0000", 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7399999999999998e-07 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193764+0000", 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193771+0000", 2024-09-16T11:13:57.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193775+0000", 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3619999999999999e-06 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311756+0000", 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11798077999999999 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311868+0000", 2024-09-16T11:13:57.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112605 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312176+0000", 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030757000000000001 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3956869279999999, 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:13:57.607 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.611 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3956681419999999, 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:13:57.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:13:57.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.616 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3956508190000001, 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:13:57.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.620 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3956333500000007, 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.621 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:13:57.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917928909999997, 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:13:57.625 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:13:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.629 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917827759999994, 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:13:57.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917728950000008, 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.634 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917637190000001, 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:13:57.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:13:57.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917476519999994, 2024-09-16T11:13:57.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:13:57.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.391731257, 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:13:57.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:13:57.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917216830000001, 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.651 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:13:57.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3917121469999998, 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:13:57.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50319 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3876657530000003, 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118443188, 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50319 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201804+0000", 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-06 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201807+0000", 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:13:57.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201809+0000", 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320222+0000", 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118412609 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320245+0000", 2024-09-16T11:13:57.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3805999999999999e-05 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3876554900000002, 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3876461080000002, 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.667 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:13:57.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:13:57.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3876374499999997, 2024-09-16T11:13:57.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:13:57.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:57.675 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3876282159999995, 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:57.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:57.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:57.679 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:57.681 INFO:teuthology.orchestra.run.smithi162.stdout:2448 trunc from 0xa143da3 to 0xdc897 2024-09-16T11:13:57.681 INFO:teuthology.orchestra.run.smithi162.stdout:2449 read 0xb952 thru 0xbd29 (0x3d8 bytes) 2024-09-16T11:13:57.681 INFO:teuthology.orchestra.run.smithi162.stdout:2450 read 0x74f24 thru 0x7696e (0x1a4b bytes) 2024-09-16T11:13:57.681 INFO:teuthology.orchestra.run.smithi162.stdout:2451 read 0xc97f0 thru 0xd66c1 (0xced2 bytes) 2024-09-16T11:13:57.682 INFO:teuthology.orchestra.run.smithi162.stdout:2455 write 0x6894b5 thru 0x6924eb (0x9037 bytes) 2024-09-16T11:13:57.682 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:13:57.682 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:57.673+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:57.683 INFO:teuthology.orchestra.run.smithi162.stdout:2456 writesame 0x6f1a8f7 thru 0x6f28370 data_size 0xaa(0xda7a bytes) 2024-09-16T11:13:57.683 INFO:teuthology.orchestra.run.smithi162.stdout:2457 write 0x1ce4658 thru 0x1ceb1cb (0x6b74 bytes) 2024-09-16T11:13:57.690 INFO:teuthology.orchestra.run.smithi162.stdout:2458 read 0x50a9c45 thru 0x50adc8b (0x4047 bytes) 2024-09-16T11:13:57.690 INFO:teuthology.orchestra.run.smithi162.stdout:2460 read 0xdf7150 thru 0xdf7862 (0x713 bytes) 2024-09-16T11:13:57.690 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.699 INFO:teuthology.orchestra.run.smithi162.stdout:2465 trunc from 0x6f28371 to 0x168f3e2 2024-09-16T11:13:57.711 INFO:teuthology.orchestra.run.smithi162.stdout:2468 read 0x9236d9 thru 0x924271 (0xb99 bytes) 2024-09-16T11:13:57.730 INFO:teuthology.orchestra.run.smithi162.stdout:2469 writesame 0x518646c thru 0x518f712 data_size 0x1(0x92a7 bytes) 2024-09-16T11:13:57.732 INFO:teuthology.orchestra.run.smithi162.stdout:2470 read 0x11155ce thru 0x1117cc1 (0x26f4 bytes) 2024-09-16T11:13:57.736 INFO:teuthology.orchestra.run.smithi162.stdout:2471 trunc from 0x518f713 to 0x28c361b 2024-09-16T11:13:57.750 INFO:teuthology.orchestra.run.smithi162.stdout:2472 punch from 0x1077e19 to 0x1085660, (0xd847 bytes) 2024-09-16T11:13:57.770 INFO:teuthology.orchestra.run.smithi162.stdout:2473 trunc from 0x28c361b to 0xda74f09 2024-09-16T11:13:57.781 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:57.776+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:57.782 INFO:teuthology.orchestra.run.smithi162.stdout:2474 punch from 0x8fb375 to 0x907c12, (0xc89d bytes) 2024-09-16T11:13:57.782 INFO:teuthology.orchestra.run.smithi162.stdout:2476 write 0xd8bc3e3 thru 0xd8c2084 (0x5ca2 bytes) 2024-09-16T11:13:57.789 INFO:teuthology.orchestra.run.smithi162.stdout:2477 writesame 0x882bf0b thru 0x8837cc5 data_size 0x1(0xbdbb bytes) 2024-09-16T11:13:57.793 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:57.793 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:57.793 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:13:57.793 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:57.804 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:13:57.845 INFO:teuthology.orchestra.run.smithi162.stdout:2478 trunc from 0xda74f09 to 0x4edb542 2024-09-16T11:13:57.877 INFO:teuthology.orchestra.run.smithi162.stdout:2479 write 0x5b5b116 thru 0x5b629d5 (0x78c0 bytes) 2024-09-16T11:13:57.880 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:57.880 INFO:teuthology.orchestra.run.smithi162.stdout:2482 writesame 0x54f9357 thru 0x5506538 data_size 0x5a(0xd1e2 bytes) 2024-09-16T11:13:57.882 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:57.876+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:57.884 INFO:teuthology.orchestra.run.smithi162.stdout:2483 read 0x268f753 thru 0x2697542 (0x7df0 bytes) 2024-09-16T11:13:57.899 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:57.899 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:13:57.899 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:13:57.899 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:13:57.899 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:57.902 INFO:teuthology.orchestra.run.smithi162.stdout:2484 trunc from 0x5b629d6 to 0xe5d6f69 2024-09-16T11:13:57.909 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:13:57.914 INFO:teuthology.orchestra.run.smithi162.stdout:2485 writesame 0xd8ba402 thru 0xd8ba4ea data_size 0x1(0xe9 bytes) 2024-09-16T11:13:57.918 INFO:teuthology.orchestra.run.smithi162.stdout:2486 writesame 0x271f330 thru 0x272a0b3 data_size 0x2(0xad84 bytes) 2024-09-16T11:13:57.936 INFO:teuthology.orchestra.run.smithi162.stdout:2487 punch from 0xb7dad71 to 0xb7e081a, (0x5aa9 bytes) 2024-09-16T11:13:57.936 INFO:teuthology.orchestra.run.smithi162.stdout:2488 punch from 0x2558f5f to 0x25686bd, (0xf75e bytes) 2024-09-16T11:13:57.937 INFO:teuthology.orchestra.run.smithi162.stdout:2489 read 0x7fe104c thru 0x7fe376d (0x2722 bytes) 2024-09-16T11:13:57.937 INFO:teuthology.orchestra.run.smithi162.stdout:2490 read 0x4dce986 thru 0x4ddbfd6 (0xd651 bytes) 2024-09-16T11:13:57.937 INFO:teuthology.orchestra.run.smithi162.stdout:2491 write 0x6aaa9eb thru 0x6aafb18 (0x512e bytes) 2024-09-16T11:13:57.943 INFO:teuthology.orchestra.run.smithi162.stdout:2493 read 0xb200a0c thru 0xb202298 (0x188d bytes) 2024-09-16T11:13:57.967 INFO:teuthology.orchestra.run.smithi162.stdout:2494 trunc from 0xe5d6f69 to 0x253e75 2024-09-16T11:13:57.983 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:57.977+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:57.992 INFO:teuthology.orchestra.run.smithi162.stdout:2495 punch from 0x1c3cc4 to 0x1c7fd5, (0x4311 bytes) 2024-09-16T11:13:58.007 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 68.227074815999998, 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:13:58.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:13:58.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.839149116000002, 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:13:58.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838997785999993, 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:13:58.020 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:13:58.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:13:58.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838946300000003, 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:13:58.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.031 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838876151999997, 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:13:58.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:13:58.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838703682000002, 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:13:58.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:13:58.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:58.043 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838645360000001, 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:13:58.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:13:58.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.049 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838493240000005, 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:58.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:13:58.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838314721000003, 2024-09-16T11:13:58.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:13:58.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.061 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.838281748, 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:13:58.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.286116256, 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:13:58.067 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:13:58.072 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.286094023000004, 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.073 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.285965226000002, 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:58.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.285906881000002, 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:58.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:13:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.285852492000004, 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:13:58.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.285718184999993, 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.096 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:13:58.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:13:58.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.504620665000004, 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011970313, 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 91 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498474+0000", 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498476+0000", 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.674e-06 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498485+0000", 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6930000000000002e-06 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498503+0000", 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7643e-05 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498518+0000", 2024-09-16T11:13:58.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4993e-05 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.503966+0000", 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005447894 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504151+0000", 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018552299999999999 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.504404+0000", 2024-09-16T11:13:58.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025254699999999999 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509080+0000", 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046758420000000004 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510320+0000", 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0012405840000000001 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510391+0000", 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0706999999999999e-05 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510444+0000", 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3213e-05 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:58.107 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.504562816000004, 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:13:58.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:13:58.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:13:58.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.504475503999998, 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.113 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:13:58.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:13:58.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.504237376000006, 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:13:58.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:13:58.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:13:58.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:13:58.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:13:58.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:13:58.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:13:58.125 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:13:58.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:13:58.128 INFO:teuthology.orchestra.run.smithi162.stdout:2496 write 0xc5296b2 thru 0xc52f9c9 (0x6318 bytes) 2024-09-16T11:13:58.128 INFO:teuthology.orchestra.run.smithi162.stdout:2498 punch from 0x2b66a30 to 0x2b6a859, (0x3e29 bytes) 2024-09-16T11:13:58.128 INFO:teuthology.orchestra.run.smithi162.stdout:2499 writesame 0xb112994 thru 0xb12206a data_size 0x1(0xf6d7 bytes) 2024-09-16T11:13:58.128 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.128 INFO:teuthology.orchestra.run.smithi162.stdout:2501 writesame 0x312a0da thru 0x3131f6f data_size 0xb(0x7e96 bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2502 read 0xa7cf308 thru 0xa7de87f (0xf578 bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2503 punch from 0xb8016a to 0xb8061c, (0x4b2 bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2506 read 0xb984c16 thru 0xb98b260 (0x664b bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2507 punch from 0x4a45731 to 0x4a49838, (0x4107 bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2508 read 0xe03d84 thru 0xe09812 (0x5a8f bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2509 write 0x9f3ec78 thru 0x9f3edb1 (0x13a bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2510 write 0x75b7f6e thru 0x75bbda0 (0x3e33 bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2511 punch from 0x402766b to 0x40336d9, (0xc06e bytes) 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:2512 trunc from 0xc52f9ca to 0xa9b449 2024-09-16T11:13:58.129 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.129 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:58.120+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:58.145 INFO:teuthology.orchestra.run.smithi162.stdout:2515 writesame 0x3f5507f thru 0x3f5b3fc data_size 0x2d(0x637e bytes) 2024-09-16T11:13:58.147 INFO:teuthology.orchestra.run.smithi162.stdout:2516 write 0x2431c67 thru 0x24366f0 (0x4a8a bytes) 2024-09-16T11:13:58.149 INFO:teuthology.orchestra.run.smithi162.stdout:2517 write 0x2bc036c thru 0x2bcad22 (0xa9b7 bytes) 2024-09-16T11:13:58.175 INFO:teuthology.orchestra.run.smithi162.stdout:2518 writesame 0x9088426 thru 0x90915a6 data_size 0x1(0x9181 bytes) 2024-09-16T11:13:58.181 INFO:teuthology.orchestra.run.smithi162.stdout:2519 trunc from 0x90915a7 to 0x512d365 2024-09-16T11:13:58.196 INFO:teuthology.orchestra.run.smithi162.stdout:2520 read 0xdeaaaa thru 0xdf4250 (0x97a7 bytes) 2024-09-16T11:13:58.225 INFO:teuthology.orchestra.run.smithi162.stdout:2521 write 0xc994106 thru 0xc9a06ac (0xc5a7 bytes) 2024-09-16T11:13:58.227 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:13:58.221+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:13:58.232 INFO:teuthology.orchestra.run.smithi162.stdout:2522 read 0x2980ffe thru 0x298c956 (0xb959 bytes) 2024-09-16T11:13:58.232 INFO:teuthology.orchestra.run.smithi162.stdout:2523 read 0x861b54f thru 0x8626fce (0xba80 bytes) 2024-09-16T11:13:58.234 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:58.235 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:58.235 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:58.247 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:13:58.253 INFO:teuthology.orchestra.run.smithi162.stdout:2524 trunc from 0xc9a06ad to 0x451a8b 2024-09-16T11:13:58.278 INFO:teuthology.orchestra.run.smithi162.stdout:2525 read 0xb27cf thru 0xb7475 (0x4ca7 bytes) 2024-09-16T11:13:58.304 INFO:teuthology.orchestra.run.smithi162.stdout:2526 write 0x5a62130 thru 0x5a6fbfb (0xdacc bytes) 2024-09-16T11:13:58.307 INFO:teuthology.orchestra.run.smithi162.stdout:2527 read 0x4b0cec9 thru 0x4b0fe1d (0x2f55 bytes) 2024-09-16T11:13:58.308 INFO:teuthology.orchestra.run.smithi162.stdout:2529 write 0x3fec743 thru 0x3ffa7e5 (0xe0a3 bytes) 2024-09-16T11:13:58.310 INFO:teuthology.orchestra.run.smithi162.stdout:2530 punch from 0x55f3a2b to 0x55fe71b, (0xacf0 bytes) 2024-09-16T11:13:58.315 INFO:teuthology.orchestra.run.smithi162.stdout:2531 trunc from 0x5a6fbfc to 0x2988f58 2024-09-16T11:13:58.329 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:58.320+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:58.332 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.356 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:58.356 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:58.356 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:58.356 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:58.356 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:58.359 INFO:teuthology.orchestra.run.smithi162.stdout:2534 write 0xb395b2c thru 0xb3a32ce (0xd7a3 bytes) 2024-09-16T11:13:58.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:13:58.367 INFO:teuthology.orchestra.run.smithi162.stdout:2535 trunc from 0xb3a32cf to 0xa64ece2 2024-09-16T11:13:58.383 INFO:teuthology.orchestra.run.smithi162.stdout:2536 read 0x462c7b1 thru 0x46336f8 (0x6f48 bytes) 2024-09-16T11:13:58.384 INFO:teuthology.orchestra.run.smithi162.stdout:2537 write 0x1078a34 thru 0x10823f4 (0x99c1 bytes) 2024-09-16T11:13:58.387 INFO:teuthology.orchestra.run.smithi162.stdout:2541 write 0x6d98df4 thru 0x6d9ca39 (0x3c46 bytes) 2024-09-16T11:13:58.394 INFO:teuthology.orchestra.run.smithi162.stdout:2542 read 0x9f027bc thru 0x9f0a6de (0x7f23 bytes) 2024-09-16T11:13:58.394 INFO:teuthology.orchestra.run.smithi162.stdout:2543 punch from 0x394dedd to 0x3957708, (0x982b bytes) 2024-09-16T11:13:58.416 INFO:teuthology.orchestra.run.smithi162.stdout:2544 write 0xee1bc1c thru 0xee27cb1 (0xc096 bytes) 2024-09-16T11:13:58.423 INFO:teuthology.orchestra.run.smithi162.stdout:2545 write 0x402f3ce thru 0x403c829 (0xd45c bytes) 2024-09-16T11:13:58.427 INFO:teuthology.orchestra.run.smithi162.stdout:2547 read 0x1a5c08f thru 0x1a64700 (0x8672 bytes) 2024-09-16T11:13:58.427 INFO:teuthology.orchestra.run.smithi162.stdout:2548 write 0xd1f01be thru 0xd1f87e8 (0x862b bytes) 2024-09-16T11:13:58.429 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:58.421+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:58.438 INFO:teuthology.orchestra.run.smithi162.stdout:2550 punch from 0xe66e9b4 to 0xe677812, (0x8e5e bytes) 2024-09-16T11:13:58.439 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:13:58.451 INFO:teuthology.orchestra.run.smithi162.stdout:2551 trunc from 0xee27cb2 to 0x695321e 2024-09-16T11:13:58.478 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.301485470999999, 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:13:58.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.301407727999999, 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:13:58.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.299041712999999, 2024-09-16T11:13:58.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:13:58.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.492 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.299022196999999, 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:13:58.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.497 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.297884693, 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:58.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:13:58.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.297868793999999, 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.502 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:13:58.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:58.506 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.29784454, 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:13:58.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:13:58.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.297829281, 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.511 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:13:58.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:58.515 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.297813961999999, 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:58.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:13:58.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295791557999999, 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:13:58.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295777060000001, 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:58.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:13:58.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295762727, 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:13:58.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.29574867, 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:13:58.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:13:58.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295734813999999, 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:13:58.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:13:58.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295720939000001, 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:13:58.542 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:13:58.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:13:58.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295707172, 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.547 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:13:58.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.551 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295659663, 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:13:58.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.295373845, 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:13:58.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.202710029, 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:13:58.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:13:58.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:58.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.202560847000001, 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:13:58.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:58.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.572 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:58.573 INFO:teuthology.orchestra.run.smithi162.stdout:2552 write 0x5f7ee69 thru 0x5f805ff (0x1797 bytes) 2024-09-16T11:13:58.574 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2553 writesame 0x863a169 thru 0x8646acf data_size 0x1(0xc967 bytes) 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2554 write 0x19ced82 thru 0x19d8f50 (0xa1cf bytes) 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2555 read 0x365da6e thru 0x366c3f3 (0xe986 bytes) 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2556 read 0x7f72dba thru 0x7f753ac (0x25f3 bytes) 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2557 trunc from 0x8646ad0 to 0xb886f4f 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2558 read 0x682123 thru 0x68b409 (0x92e7 bytes) 2024-09-16T11:13:58.575 INFO:teuthology.orchestra.run.smithi162.stdout:2559 read 0x769e0dc thru 0x76a7aac (0x99d1 bytes) 2024-09-16T11:13:58.576 INFO:teuthology.orchestra.run.smithi162.stdout:2560 read 0x2cc5b90 thru 0x2cd2e2e (0xd29f bytes) 2024-09-16T11:13:58.576 INFO:teuthology.orchestra.run.smithi162.stdout:2562 punch from 0x934df08 to 0x935c5c4, (0xe6bc bytes) 2024-09-16T11:13:58.576 INFO:teuthology.orchestra.run.smithi162.stdout:2564 read 0x9c96e7f thru 0x9ca0b35 (0x9cb7 bytes) 2024-09-16T11:13:58.576 INFO:teuthology.orchestra.run.smithi162.stdout:2565 trunc from 0xb886f4f to 0x34bb6ab 2024-09-16T11:13:58.597 INFO:teuthology.orchestra.run.smithi162.stdout:2566 write 0x8dd3d8c thru 0x8dd6052 (0x22c7 bytes) 2024-09-16T11:13:58.598 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:13:58.592+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:13:58.625 INFO:teuthology.orchestra.run.smithi162.stdout:2567 write 0xca81b5a thru 0xca83cb1 (0x2158 bytes) 2024-09-16T11:13:58.632 INFO:teuthology.orchestra.run.smithi162.stdout:2568 read 0x4ac4d9b thru 0x4acc372 (0x75d8 bytes) 2024-09-16T11:13:58.645 INFO:teuthology.orchestra.run.smithi162.stdout:2570 write 0xca88465 thru 0xca8e0cc (0x5c68 bytes) 2024-09-16T11:13:58.649 INFO:teuthology.orchestra.run.smithi162.stdout:2571 punch from 0xef1756 to 0xef41d6, (0x2a80 bytes) 2024-09-16T11:13:58.649 INFO:teuthology.orchestra.run.smithi162.stdout:2572 read 0xb3cc7a0 thru 0xb3d5bbe (0x941f bytes) 2024-09-16T11:13:58.649 INFO:teuthology.orchestra.run.smithi162.stdout:2573 read 0xb0bf2a4 thru 0xb0c6dbf (0x7b1c bytes) 2024-09-16T11:13:58.649 INFO:teuthology.orchestra.run.smithi162.stdout:2574 read 0x3fe0347 thru 0x3ff02cd (0xff87 bytes) 2024-09-16T11:13:58.650 INFO:teuthology.orchestra.run.smithi162.stdout:2577 punch from 0xdaac70 to 0xdb2fb0, (0x8340 bytes) 2024-09-16T11:13:58.650 INFO:teuthology.orchestra.run.smithi162.stdout:2578 write 0xc228267 thru 0xc230b4d (0x88e7 bytes) 2024-09-16T11:13:58.657 INFO:teuthology.orchestra.run.smithi162.stdout:2580 writesame 0x3c1fe42 thru 0x3c2a27d data_size 0x2e(0xa43c bytes) 2024-09-16T11:13:58.660 INFO:teuthology.orchestra.run.smithi162.stdout:2581 read 0x67f3e1d thru 0x6802e39 (0xf01d bytes) 2024-09-16T11:13:58.664 INFO:teuthology.orchestra.run.smithi162.stdout:2584 trunc from 0xca8e0cd to 0xe32ac97 2024-09-16T11:13:58.677 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:13:58.669+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:13:58.678 INFO:teuthology.orchestra.run.smithi162.stdout:2585 write 0x8fb6470 thru 0x8fbdb01 (0x7692 bytes) 2024-09-16T11:13:58.685 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.685 INFO:teuthology.orchestra.run.smithi162.stdout:2589 write 0x554dbb2 thru 0x555207b (0x44ca bytes) 2024-09-16T11:13:58.695 INFO:teuthology.orchestra.run.smithi162.stdout:2591 trunc from 0xe32ac97 to 0x96df975 2024-09-16T11:13:58.719 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:58.719 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:58.719 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:58.722 INFO:teuthology.orchestra.run.smithi162.stdout:2592 write 0x45d2e47 thru 0x45e18b8 (0xea72 bytes) 2024-09-16T11:13:58.732 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:13:58.738 INFO:teuthology.orchestra.run.smithi162.stdout:2593 trunc from 0x96df975 to 0x1ba5cac 2024-09-16T11:13:58.764 INFO:teuthology.orchestra.run.smithi162.stdout:2594 read 0x4de247 thru 0x4df0c1 (0xe7b bytes) 2024-09-16T11:13:58.778 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:58.772+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:58.780 INFO:teuthology.orchestra.run.smithi162.stdout:2595 write 0x3e28748 thru 0x3e2a424 (0x1cdd bytes) 2024-09-16T11:13:58.810 INFO:teuthology.orchestra.run.smithi162.stdout:2596 writesame 0x965bdeb thru 0x965d7e0 data_size 0x1(0x19f6 bytes) 2024-09-16T11:13:58.819 INFO:teuthology.orchestra.run.smithi162.stdout:2599 trunc from 0x965d7e1 to 0x42edaf1 2024-09-16T11:13:58.837 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:58.837 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:58.837 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:58.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:13:58.851 INFO:teuthology.orchestra.run.smithi162.stdout:2600 writesame 0x61b30e0 thru 0x61bb82d data_size 0x45(0x874e bytes) 2024-09-16T11:13:58.871 INFO:teuthology.orchestra.run.smithi162.stdout:2601 writesame 0xaa51f30 thru 0xaa524a5 data_size 0x3(0x576 bytes) 2024-09-16T11:13:58.871 INFO:teuthology.orchestra.run.smithi162.stdout:2603 read 0x2359821 thru 0x23615e1 (0x7dc1 bytes) 2024-09-16T11:13:58.872 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.878 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:58.872+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:58.887 INFO:teuthology.orchestra.run.smithi162.stdout:2606 write 0xb98f6c6 thru 0xb996383 (0x6cbe bytes) 2024-09-16T11:13:58.894 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.895 INFO:teuthology.orchestra.run.smithi162.stdout:2609 write 0x726e2c5 thru 0x7276183 (0x7ebf bytes) 2024-09-16T11:13:58.901 INFO:teuthology.orchestra.run.smithi162.stdout:2610 punch from 0xa8a6242 to 0xa8b4ea0, (0xec5e bytes) 2024-09-16T11:13:58.901 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.901 INFO:teuthology.orchestra.run.smithi162.stdout:2612 write 0x36c07c8 thru 0x36c769c (0x6ed5 bytes) 2024-09-16T11:13:58.904 INFO:teuthology.orchestra.run.smithi162.stdout:2614 punch from 0x4c1b3b5 to 0x4c1bfa9, (0xbf4 bytes) 2024-09-16T11:13:58.904 INFO:teuthology.orchestra.run.smithi162.stdout:2615 write 0xa1ba542 thru 0xa1c95a3 (0xf062 bytes) 2024-09-16T11:13:58.907 INFO:teuthology.orchestra.run.smithi162.stdout:2616 read 0xabfc3b1 thru 0xac00653 (0x42a3 bytes) 2024-09-16T11:13:58.908 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.908 INFO:teuthology.orchestra.run.smithi162.stdout:2618 read 0x58668b5 thru 0x587095b (0xa0a7 bytes) 2024-09-16T11:13:58.909 INFO:teuthology.orchestra.run.smithi162.stdout:2619 read 0x86f60a8 thru 0x86fc622 (0x657b bytes) 2024-09-16T11:13:58.910 INFO:teuthology.orchestra.run.smithi162.stdout:2620 read 0x8195021 thru 0x81a4ff2 (0xffd2 bytes) 2024-09-16T11:13:58.911 INFO:teuthology.orchestra.run.smithi162.stdout:2621 read 0x3fe5aa8 thru 0x3fea00f (0x4568 bytes) 2024-09-16T11:13:58.912 INFO:teuthology.orchestra.run.smithi162.stdout:2622 read 0x2ae57d thru 0x2b42f6 (0x5d7a bytes) 2024-09-16T11:13:58.913 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:58.913 INFO:teuthology.orchestra.run.smithi162.stdout:2624 write 0xa515fec thru 0xa51e915 (0x892a bytes) 2024-09-16T11:13:58.916 INFO:teuthology.orchestra.run.smithi162.stdout:2625 writesame 0x912b8 thru 0x9c6c9 data_size 0x75(0xb412 bytes) 2024-09-16T11:13:58.938 INFO:teuthology.orchestra.run.smithi162.stdout:2626 write 0xdb59578 thru 0xdb5d7b5 (0x423e bytes) 2024-09-16T11:13:58.948 INFO:teuthology.orchestra.run.smithi162.stdout:2627 read 0xd264350 thru 0xd26444f (0x100 bytes) 2024-09-16T11:13:58.949 INFO:teuthology.orchestra.run.smithi162.stdout:2632 punch from 0x7251fbb to 0x726048c, (0xe4d1 bytes) 2024-09-16T11:13:58.949 INFO:teuthology.orchestra.run.smithi162.stdout:2633 write 0x432c2be thru 0x43319ce (0x5711 bytes) 2024-09-16T11:13:58.950 INFO:teuthology.orchestra.run.smithi162.stdout:2634 punch from 0x34bf1bb to 0x34c8e3a, (0x9c7f bytes) 2024-09-16T11:13:58.956 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:58.956 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50255 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794375807, 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063800420999999996, 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50255 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159697+0000", 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2e-07 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159699+0000", 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.114e-06 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159705+0000", 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9970000000000004e-06 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159709+0000", 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223159+0000", 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063450079000000006 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223206+0000", 2024-09-16T11:13:58.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7148000000000003e-05 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223497+0000", 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000290434 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794359223000001, 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:13:58.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:13:58.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794324168999999, 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:13:58.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:13:58.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794307735, 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794291426999999, 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:13:58.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:58.978 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794275860999999, 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:13:58.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.793541367, 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:13:58.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.793525339, 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:13:58.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.793515509000001, 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:13:58.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.790205313, 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:58.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:58.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:58.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:58.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.790181963, 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:13:59.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.790162663, 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.005 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:13:59.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.790145181, 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:13:59.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:13:59.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.790114601999999, 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:13:59.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.789992785000001, 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:13:59.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.022 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.789551615000001, 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:59.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:13:59.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788920743, 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:13:59.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.031 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788892461, 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:13:59.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:13:59.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788877233000001, 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:59.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:13:59.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:13:59.040 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788864330999999, 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:59.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.044 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.045 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:59.046 INFO:teuthology.orchestra.run.smithi162.stdout:2636 trunc from 0xdb5d7b6 to 0x86b916e 2024-09-16T11:13:59.047 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2639 write 0x83f6c89 thru 0x83f6d66 (0xde bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2640 read 0x25d9331 thru 0x25e7287 (0xdf57 bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2642 writesame 0x255910 thru 0x264669 data_size 0x7b(0xed5a bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2644 read 0x4ef06b4 thru 0x4eff09c (0xe9e9 bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2645 punch from 0x65feae8 to 0x660c04a, (0xd562 bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2648 write 0x2e9b098 thru 0x2ea8791 (0xd6fa bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2650 write 0x8d7049e thru 0x8d79c7e (0x97e1 bytes) 2024-09-16T11:13:59.048 INFO:teuthology.orchestra.run.smithi162.stdout:2651 writesame 0x3bb55c0 thru 0x3bc0876 data_size 0x1(0xb2b7 bytes) 2024-09-16T11:13:59.048 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:59.040+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:59.050 INFO:teuthology.orchestra.run.smithi162.stdout:2653 read 0x4dfb872 thru 0x4e09a61 (0xe1f0 bytes) 2024-09-16T11:13:59.052 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:59.055 INFO:teuthology.orchestra.run.smithi162.stdout:2655 trunc from 0x8d79c7f to 0x78569c7 2024-09-16T11:13:59.072 INFO:teuthology.orchestra.run.smithi162.stdout:2657 punch from 0x1d89225 to 0x1d8bc0c, (0x29e7 bytes) 2024-09-16T11:13:59.073 INFO:teuthology.orchestra.run.smithi162.stdout:2658 write 0x4b663ee thru 0x4b67ae2 (0x16f5 bytes) 2024-09-16T11:13:59.094 INFO:teuthology.orchestra.run.smithi162.stdout:2660 write 0xa33fc8e thru 0xa3476db (0x7a4e bytes) 2024-09-16T11:13:59.100 INFO:teuthology.orchestra.run.smithi162.stdout:2661 punch from 0x709b24c to 0x70a5574, (0xa328 bytes) 2024-09-16T11:13:59.117 INFO:teuthology.orchestra.run.smithi162.stdout:2663 write 0xbc23c93 thru 0xbc2c93a (0x8ca8 bytes) 2024-09-16T11:13:59.136 INFO:teuthology.orchestra.run.smithi162.stdout:2664 writesame 0xc9fcbf2 thru 0xca03e43 data_size 0x1(0x7252 bytes) 2024-09-16T11:13:59.136 INFO:teuthology.orchestra.run.smithi162.stdout:2665 write 0xb292cc9 thru 0xb2a028c (0xd5c4 bytes) 2024-09-16T11:13:59.140 INFO:teuthology.orchestra.run.smithi162.stdout:2667 writesame 0x20f3e7e thru 0x20f742d data_size 0x8(0x35b0 bytes) 2024-09-16T11:13:59.148 INFO:teuthology.orchestra.run.smithi162.stdout:2668 read 0xbf0d48e thru 0xbf13228 (0x5d9b bytes) 2024-09-16T11:13:59.149 INFO:teuthology.orchestra.run.smithi162.stdout:2669 write 0x844fca7 thru 0x84559e1 (0x5d3b bytes) 2024-09-16T11:13:59.149 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:59.140+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:59.150 INFO:teuthology.orchestra.run.smithi162.stdout:2670 read 0xac86390 thru 0xac8bfff (0x5c70 bytes) 2024-09-16T11:13:59.151 INFO:teuthology.orchestra.run.smithi162.stdout:2673 read 0x96305b0 thru 0x9630a0a (0x45b bytes) 2024-09-16T11:13:59.152 INFO:teuthology.orchestra.run.smithi162.stdout:2674 read 0x310b913 thru 0x310fe37 (0x4525 bytes) 2024-09-16T11:13:59.153 INFO:teuthology.orchestra.run.smithi162.stdout:2676 write 0xb1e33c8 thru 0xb1ec7e2 (0x941b bytes) 2024-09-16T11:13:59.157 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:59.157 INFO:teuthology.orchestra.run.smithi162.stdout:2680 read 0x3828b22 thru 0x382e1c5 (0x56a4 bytes) 2024-09-16T11:13:59.158 INFO:teuthology.orchestra.run.smithi162.stdout:2681 writesame 0x308fd4 thru 0x310c49 data_size 0x59(0x7c76 bytes) 2024-09-16T11:13:59.159 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:59.159 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:59.159 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:59.159 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:59.162 INFO:teuthology.orchestra.run.smithi162.stdout:2682 writesame 0xb1fefde thru 0xb20117d data_size 0x10(0x21a0 bytes) 2024-09-16T11:13:59.167 INFO:teuthology.orchestra.run.smithi162.stdout:2683 read 0xc48de6c thru 0xc49b58f (0xd724 bytes) 2024-09-16T11:13:59.167 INFO:teuthology.orchestra.run.smithi162.stdout:2686 read 0x3ef6e0e thru 0x3f04883 (0xda76 bytes) 2024-09-16T11:13:59.168 INFO:teuthology.orchestra.run.smithi162.stdout:2687 write 0x69cdf9d thru 0x69d8463 (0xa4c7 bytes) 2024-09-16T11:13:59.169 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:13:59.172 INFO:teuthology.orchestra.run.smithi162.stdout:2689 write 0x349f70d thru 0x34a7707 (0x7ffb bytes) 2024-09-16T11:13:59.175 INFO:teuthology.orchestra.run.smithi162.stdout:2690 write 0x1dc7dae thru 0x1dcfd4b (0x7f9e bytes) 2024-09-16T11:13:59.178 INFO:teuthology.orchestra.run.smithi162.stdout:2691 write 0xbe93b74 thru 0xbe9d3b5 (0x9842 bytes) 2024-09-16T11:13:59.181 INFO:teuthology.orchestra.run.smithi162.stdout:2692 write 0x4c660e thru 0x4c7eea (0x18dd bytes) 2024-09-16T11:13:59.184 INFO:teuthology.orchestra.run.smithi162.stdout:2694 punch from 0x6d2eb31 to 0x6d3acf2, (0xc1c1 bytes) 2024-09-16T11:13:59.184 INFO:teuthology.orchestra.run.smithi162.stdout:2695 clone 19 order 22 su 32768 sc 2 2024-09-16T11:13:59.250 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:13:59.244+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:13:59.266 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:13:59.266 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:13:59.266 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:59.275 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:13:59.350 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:59.344+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 64.586250054999994, 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:13:59.418 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:13:59.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.295675285000002, 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:13:59.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:13:59.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.429 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.287266997000003, 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:13:59.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:13:59.433 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:59.435 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:13:59.442 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:13:59.451 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:59.445+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:59.553 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:13:59.548+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:13:59.564 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:13:59.564 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:13:59.564 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:13:59.564 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:13:59.572 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:13:59.613 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone18 from 0xca03e44 (overlap 0xdc897) to 0xc67bb 2024-09-16T11:13:59.614 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:13:59.636 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:13:59.654 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:59.645+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:59.655 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:13:59.673 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:13:59.677 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:59.677 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:59.687 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:13:59.697 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:13:59.720 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:13:59.744 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:13:59.755 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:13:59.749+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:13:59.765 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:13:59.769 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:13:59.770 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:13:59.782 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-16T11:13:59.793 INFO:teuthology.orchestra.run.smithi162.stdout:2696 write 0x18ea1e6 thru 0x18f1da2 (0x7bbd bytes) 2024-09-16T11:13:59.804 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:59.804 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:59.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:13:59.857 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:13:59.852+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:13:59.860 INFO:teuthology.orchestra.run.smithi162.stdout:2698 write 0x1b479b9 thru 0x1b4d9fe (0x6046 bytes) 2024-09-16T11:13:59.863 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:13:59.864 INFO:teuthology.orchestra.run.smithi162.stdout:2701 write 0x494db29 thru 0x4959cdf (0xc1b7 bytes) 2024-09-16T11:13:59.895 INFO:teuthology.orchestra.run.smithi162.stdout:2702 read 0x5e9d07d thru 0x5ea950e (0xc492 bytes) 2024-09-16T11:13:59.899 INFO:teuthology.orchestra.run.smithi162.stdout:2703 write 0x2a735ce thru 0x2a7af00 (0x7933 bytes) 2024-09-16T11:13:59.938 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:13:59.938 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:13:59.947 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:13:59.957 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:13:59.949+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:13:59.963 INFO:teuthology.orchestra.run.smithi162.stdout:2705 punch from 0x9cfef26 to 0x9d0ce6e, (0xdf48 bytes) 2024-09-16T11:13:59.964 INFO:teuthology.orchestra.run.smithi162.stdout:2709 write 0x40b3c18 thru 0x40b6a57 (0x2e40 bytes) 2024-09-16T11:13:59.992 INFO:teuthology.orchestra.run.smithi162.stdout:2710 write 0xb1a6e7b thru 0xb1b0b06 (0x9c8c bytes) 2024-09-16T11:14:00.045 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.045 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.054 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:00.058 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:00.052+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:00.060 INFO:teuthology.orchestra.run.smithi162.stdout:2711 write 0xd87d7c0 thru 0xd87dfbb (0x7fc bytes) 2024-09-16T11:14:00.065 INFO:teuthology.orchestra.run.smithi162.stdout:2712 writesame 0x65a4e2e thru 0x65acfea data_size 0x1(0x81bd bytes) 2024-09-16T11:14:00.128 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:00.128 INFO:teuthology.orchestra.run.smithi162.stdout:2714 punch from 0x492aa11 to 0x492c438, (0x1a27 bytes) 2024-09-16T11:14:00.128 INFO:teuthology.orchestra.run.smithi162.stdout:2715 punch from 0x2f0a382 to 0x2f14832, (0xa4b0 bytes) 2024-09-16T11:14:00.128 INFO:teuthology.orchestra.run.smithi162.stdout:2717 read 0x28260a8 thru 0x2827219 (0x1172 bytes) 2024-09-16T11:14:00.128 INFO:teuthology.orchestra.run.smithi162.stdout:2718 read 0x7aaaebe thru 0x7ab8137 (0xd27a bytes) 2024-09-16T11:14:00.132 INFO:teuthology.orchestra.run.smithi162.stdout:2719 write 0xad41cbe thru 0xad4ca34 (0xad77 bytes) 2024-09-16T11:14:00.159 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:00.152+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:00.159 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.159 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.170 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:00.198 INFO:teuthology.orchestra.run.smithi162.stdout:2720 write 0x737902c thru 0x7380b93 (0x7b68 bytes) 2024-09-16T11:14:00.251 INFO:teuthology.orchestra.run.smithi162.stdout:2721 write 0x70961b7 thru 0x70a491d (0xe767 bytes) 2024-09-16T11:14:00.254 INFO:teuthology.orchestra.run.smithi162.stdout:2724 write 0xe10b2 thru 0xe2a63 (0x19b2 bytes) 2024-09-16T11:14:00.260 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:00.253+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:00.273 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.273 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.286 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:00.293 INFO:teuthology.orchestra.run.smithi162.stdout:2726 trunc from 0xd87dfbc to 0xb22f304 2024-09-16T11:14:00.313 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:00.313 INFO:teuthology.orchestra.run.smithi162.stdout:2728 writesame 0x87ddcbe thru 0x87e38d1 data_size 0x4(0x5c14 bytes) 2024-09-16T11:14:00.360 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:00.352+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:00.379 INFO:teuthology.orchestra.run.smithi162.stdout:2729 punch from 0x52b5e47 to 0x52c16c3, (0xb87c bytes) 2024-09-16T11:14:00.380 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:00.380 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:00.380 INFO:teuthology.orchestra.run.smithi162.stdout:2732 write 0x76a7e1b thru 0x76aac40 (0x2e26 bytes) 2024-09-16T11:14:00.385 INFO:teuthology.orchestra.run.smithi162.stdout:2733 read 0x751e281 thru 0x7522c14 (0x4994 bytes) 2024-09-16T11:14:00.385 INFO:teuthology.orchestra.run.smithi162.stdout:2734 write 0xa2e7e06 thru 0xa2eb307 (0x3502 bytes) 2024-09-16T11:14:00.385 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.385 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.395 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:00.427 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:00.427 INFO:teuthology.orchestra.run.smithi162.stdout:2737 read 0x21c98da thru 0x21cd22b (0x3952 bytes) 2024-09-16T11:14:00.433 INFO:teuthology.orchestra.run.smithi162.stdout:2738 trunc from 0xb22f304 to 0xc5b4150 2024-09-16T11:14:00.455 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:14:00.461 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:00.453+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:00.467 INFO:teuthology.orchestra.run.smithi162.stdout:2740 trunc from 0xc5b4150 to 0x39ec68f 2024-09-16T11:14:00.499 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.499 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.509 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:00.534 INFO:teuthology.orchestra.run.smithi162.stdout:2742 trunc from 0x39ec68f to 0xbc9eeec 2024-09-16T11:14:00.541 INFO:teuthology.orchestra.run.smithi162.stdout:2745 read 0xcdd489 thru 0xceb236 (0xddae bytes) 2024-09-16T11:14:00.545 INFO:teuthology.orchestra.run.smithi162.stdout:2746 write 0x1ea6858 thru 0x1eb2964 (0xc10d bytes) 2024-09-16T11:14:00.549 INFO:teuthology.orchestra.run.smithi162.stdout:2747 read 0x337452b thru 0x337ff4d (0xba23 bytes) 2024-09-16T11:14:00.562 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:00.553+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:00.568 INFO:teuthology.orchestra.run.smithi162.stdout:2749 read 0x704b0f1 thru 0x704ff88 (0x4e98 bytes) 2024-09-16T11:14:00.568 INFO:teuthology.orchestra.run.smithi162.stdout:2750 write 0x530bf0c thru 0x5311847 (0x593c bytes) 2024-09-16T11:14:00.624 INFO:teuthology.orchestra.run.smithi162.stdout:2752 writesame 0xec10c79 thru 0xec17803 data_size 0x17(0x6b8b bytes) 2024-09-16T11:14:00.625 INFO:teuthology.orchestra.run.smithi162.stdout:2754 write 0x202273c thru 0x203162f (0xeef4 bytes) 2024-09-16T11:14:00.659 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.660 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.663 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:00.656+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:00.666 INFO:teuthology.orchestra.run.smithi162.stdout:2757 punch from 0x9d3122 to 0x9d73af, (0x428d bytes) 2024-09-16T11:14:00.666 INFO:teuthology.orchestra.run.smithi162.stdout:2758 write 0x8e9ec38 thru 0x8ea2013 (0x33dc bytes) 2024-09-16T11:14:00.669 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:00.673 INFO:teuthology.orchestra.run.smithi162.stdout:2759 punch from 0x1269dd9 to 0x126bf45, (0x216c bytes) 2024-09-16T11:14:00.673 INFO:teuthology.orchestra.run.smithi162.stdout:2761 writesame 0x60b81a7 thru 0x60bcbca data_size 0x49(0x4a24 bytes) 2024-09-16T11:14:00.678 INFO:teuthology.orchestra.run.smithi162.stdout:2763 write 0x323145 thru 0x32eac3 (0xb97f bytes) 2024-09-16T11:14:00.715 INFO:teuthology.orchestra.run.smithi162.stdout:2764 read 0xbf9127d thru 0xbf962a4 (0x5028 bytes) 2024-09-16T11:14:00.715 INFO:teuthology.orchestra.run.smithi162.stdout:2766 read 0xe2df89b thru 0xe2e99bb (0xa121 bytes) 2024-09-16T11:14:00.715 INFO:teuthology.orchestra.run.smithi162.stdout:2767 clone 20 order 22 su 262144 sc 10 2024-09-16T11:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.764 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:00.756+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:00.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:00.865 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:00.860+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:00.870 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.870 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.880 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:00.967 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:00.960+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:00.981 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:00.981 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:00.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:01.067 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:01.064+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:01.090 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.090 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.099 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:01.169 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:01.161+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:01.198 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.198 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.207 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:01.270 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:01.261+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:01.317 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.317 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.327 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:01.371 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:01.364+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:01.427 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.427 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.437 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:01.457 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:14:01.473 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:01.468+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:01.541 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.541 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.551 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:01.593 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:01.588+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:01.673 INFO:teuthology.orchestra.run.smithi162.stdout:flattening image image_client.0-clone19 2024-09-16T11:14:01.735 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.735 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.736 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:01.736 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:01.728+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:01.782 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:01.811 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:01.811 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:01.811 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-16T11:14:01.811 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:14:01.811 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:14:01.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-16T11:14:01.836 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:01.828+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:01.860 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:01.860 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:01.877 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:01.920 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:01.921 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:14:01.921 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:01.931 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:01.937 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:01.929+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:01.960 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:01.979 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:01.981 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:01.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50255 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.818519117999999, 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063800420999999996, 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50255 2024-09-16T11:14:01.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:01.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159697+0000", 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2e-07 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159699+0000", 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.114e-06 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159705+0000", 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9970000000000004e-06 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159709+0000", 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223159+0000", 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063450079000000006 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223206+0000", 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7148000000000003e-05 2024-09-16T11:14:01.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223497+0000", 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000290434 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.818502534, 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:14:01.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:14:01.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:14:01.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:01.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.818467480000001, 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:01.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:01.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:01.995 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.818451046, 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:01.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:14:01.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:14:01.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.818434738000001, 2024-09-16T11:14:02.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:14:02.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.818419172, 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:14:02.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:14:02.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.817684677999999, 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:14:02.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:02.013 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.81766865, 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:14:02.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.81765882, 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.018 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:14:02.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:14:02.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:14:02.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:14:02.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.023 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.814348624000001, 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:14:02.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.814325274, 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.029 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:14:02.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:14:02.033 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.036 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.814305974, 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.037 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:14:02.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.814288491999999, 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:14:02.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.814257913, 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.046 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:14:02.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.814136096, 2024-09-16T11:14:02.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:14:02.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.813694926, 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.059 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.813064054, 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:14:02.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.813035771999999, 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:14:02.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.813020544, 2024-09-16T11:14:02.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:14:02.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.813007642000001, 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:02.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:14:02.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:14:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.077 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.077 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.079 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:02.079 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:02.079 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:02.080 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-16T11:14:02.080 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:02.080 INFO:teuthology.orchestra.run.smithi162.stdout:2770 writesame 0x1fcac9a thru 0x1fcca1b data_size 0x3(0x1d82 bytes) 2024-09-16T11:14:02.080 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:02.081 INFO:teuthology.orchestra.run.smithi162.stdout:2771 write 0xa8fa9ef thru 0xa8fcea4 (0x24b6 bytes) 2024-09-16T11:14:02.081 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:02.072+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:02.082 INFO:teuthology.orchestra.run.smithi162.stdout:2772 write 0xbdbf177 thru 0xbdcb4b5 (0xc33f bytes) 2024-09-16T11:14:02.086 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:02.089 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:02.090 INFO:teuthology.orchestra.run.smithi162.stdout:2774 writesame 0x8bf8d20 thru 0x8bf8dee data_size 0x3(0xcf bytes) 2024-09-16T11:14:02.098 INFO:teuthology.orchestra.run.smithi162.stdout:2775 write 0x24accc7 thru 0x24b5c7d (0x8fb7 bytes) 2024-09-16T11:14:02.126 INFO:teuthology.orchestra.run.smithi162.stdout:2777 writesame 0xe0c9bbf thru 0xe0d377a data_size 0x2(0x9bbc bytes) 2024-09-16T11:14:02.149 INFO:teuthology.orchestra.run.smithi162.stdout:2778 writesame 0x70d20a9 thru 0x70dc8e4 data_size 0x6(0xa83c bytes) 2024-09-16T11:14:02.162 INFO:teuthology.orchestra.run.smithi162.stdout:2779 read 0x99a5a47 thru 0x99a6cf7 (0x12b1 bytes) 2024-09-16T11:14:02.162 INFO:teuthology.orchestra.run.smithi162.stdout:2780 read 0x1dc5ad0 thru 0x1dc9655 (0x3b86 bytes) 2024-09-16T11:14:02.165 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:02.165 INFO:teuthology.orchestra.run.smithi162.stdout:2782 read 0x8da0911 thru 0x8da1a51 (0x1141 bytes) 2024-09-16T11:14:02.167 INFO:teuthology.orchestra.run.smithi162.stdout:2784 write 0x55e0dd9 thru 0x55e79d6 (0x6bfe bytes) 2024-09-16T11:14:02.177 INFO:teuthology.orchestra.run.smithi162.stdout:2785 write 0x81c595f thru 0x81c9f7c (0x461e bytes) 2024-09-16T11:14:02.179 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:02.173+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:02.183 INFO:teuthology.orchestra.run.smithi162.stdout:2786 read 0x53e468e thru 0x53f0336 (0xbca9 bytes) 2024-09-16T11:14:02.184 INFO:teuthology.orchestra.run.smithi162.stdout:2787 read 0x41a6419 thru 0x41b3455 (0xd03d bytes) 2024-09-16T11:14:02.184 INFO:teuthology.orchestra.run.smithi162.stdout:2788 writesame 0x8e6947e thru 0x8e6f879 data_size 0x6c(0x63fc bytes) 2024-09-16T11:14:02.186 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:02.186 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:02.186 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:02.186 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.189 INFO:teuthology.orchestra.run.smithi162.stdout:2789 punch from 0xc8b897d to 0xc8c5bb7, (0xd23a bytes) 2024-09-16T11:14:02.189 INFO:teuthology.orchestra.run.smithi162.stdout:2790 punch from 0x629da14 to 0x62a7cf1, (0xa2dd bytes) 2024-09-16T11:14:02.189 INFO:teuthology.orchestra.run.smithi162.stdout:2791 write 0xc3b94d7 thru 0xc3bb4b6 (0x1fe0 bytes) 2024-09-16T11:14:02.196 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:02.199 INFO:teuthology.orchestra.run.smithi162.stdout:2795 writesame 0x36518ac thru 0x365f827 data_size 0x2(0xdf7c bytes) 2024-09-16T11:14:02.230 INFO:teuthology.orchestra.run.smithi162.stdout:2796 writesame 0xcb5db4d thru 0xcb62c69 data_size 0x1(0x511d bytes) 2024-09-16T11:14:02.272 INFO:teuthology.orchestra.run.smithi162.stdout:2797 trunc from 0xec17804 to 0x1ff7658 2024-09-16T11:14:02.280 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:02.272+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:02.307 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:02.308 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:02.308 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.325 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:02.348 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:02.348 INFO:teuthology.orchestra.run.smithi162.stdout:2800 clone 21 order 20 su 16384 sc 2 2024-09-16T11:14:02.393 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:02.388+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.613758068999999, 2024-09-16T11:14:02.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:14:02.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:14:02.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.323183299, 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:14:02.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:14:02.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.314775011000002, 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:14:02.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:14:02.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:14:02.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:14:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.461 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.461 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.462 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:02.481 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:02.476+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:02.565 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:02.565 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:02.565 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.575 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:02.576 INFO:teuthology.orchestra.run.smithi162.stdout:leaving image image_client.0-clone20 intact 2024-09-16T11:14:02.582 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:02.581+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:02.672 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:02.672 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:02.672 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.676 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-16T11:14:02.681 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:02.682 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:02.676+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:02.784 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49775 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1286144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.648303946, 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050091745, 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49775 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133156+0000", 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1179999999999999e-06 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133159+0000", 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133178+0000", 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8692e-05 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.182940+0000", 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049762038000000001 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183007+0000", 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7082000000000006e-05 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183247+0000", 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239581 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.64827901, 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.648145267, 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:02.797 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.648126878999999, 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:02.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:02.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.647826122, 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:02.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.647796417, 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.806 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.647783019, 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:02.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.814 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.647769215, 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.818 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.647394216, 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:02.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:02.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:02.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50035 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.633865908000001, 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049942324000000003, 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50035 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147595+0000", 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9769999999999999e-06 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147601+0000", 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147605+0000", 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197156+0000", 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049550679 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197231+0000", 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5853999999999996e-05 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197535+0000", 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000303861 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.633850185, 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:02.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:02.831 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.633828611, 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:02.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.835 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.633812111999999, 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.633796036, 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:02.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.844 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.633412462000001, 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:02.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:02.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50291 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.621374389, 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049900104000000001, 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50291 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160085+0000", 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7000000000000004e-07 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160089+0000", 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6679999999999999e-06 2024-09-16T11:14:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160100+0000", 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1311999999999999e-05 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160105+0000", 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.583e-06 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209651+0000", 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049546305999999998 2024-09-16T11:14:02.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209730+0000", 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9295999999999994e-05 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209985+0000", 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254169 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50292 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.621349563999999, 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050216150000000001, 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50292 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160110+0000", 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160117+0000", 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.362e-06 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160124+0000", 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160126+0000", 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210014+0000", 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049887881000000002 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210108+0000", 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3356000000000005e-05 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210325+0000", 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021782400000000001 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.621330284999999, 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:02.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.864 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.621322083999999, 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:02.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.868 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.621314062, 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:02.869 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:02.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:02.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:02.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:02.873 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:02.873 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:02.776+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:02.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:02.884 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:02.876+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:02.985 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:02.985 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:02.985 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:02.985 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:02.985 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:02.977+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:02.994 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:03.086 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:03.077+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:03.097 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:03.098 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.107 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:03.126 INFO:teuthology.orchestra.run.smithi162.stdout:2801 read 0x1f037a0 thru 0x1f107c1 (0xd022 bytes) 2024-09-16T11:14:03.136 INFO:teuthology.orchestra.run.smithi162.stdout:2802 read 0x641307 thru 0x64ae80 (0x9b7a bytes) 2024-09-16T11:14:03.155 INFO:teuthology.orchestra.run.smithi162.stdout:2804 read 0x2c4bff thru 0x2cfa54 (0xae56 bytes) 2024-09-16T11:14:03.166 INFO:teuthology.orchestra.run.smithi162.stdout:2805 trunc from 0x1ff7658 to 0x1621f52 2024-09-16T11:14:03.187 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:03.181+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:03.240 INFO:teuthology.orchestra.run.smithi162.stdout:2806 read 0x6f927a thru 0x70059c (0x7323 bytes) 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15969 4.d 4:b69afd92:::rbd_data.11b58a761a59.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.168836950999999, 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044609673000000002, 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.251 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15969 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079092+0000", 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.17e-06 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079104+0000", 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079140+0000", 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5648000000000002e-05 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079152+0000", 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2636999999999999e-05 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079270+0000", 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011812200000000001 2024-09-16T11:14:03.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079499+0000", 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022818 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.082238+0000", 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027396949999999999 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123504+0000", 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041265556000000002 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123526+0000", 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1971999999999999e-05 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123623+0000", 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7347000000000002e-05 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123700+0000", 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6786000000000006e-05 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15970 4.d 4:b7b5bf19:::rbd_data.11b58a761a59.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.168532289000002, 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044492102999999998, 2024-09-16T11:14:03.257 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15970 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079397+0000", 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.793e-06 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079405+0000", 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079421+0000", 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6201e-05 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079429+0000", 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.092788+0000", 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013358761 2024-09-16T11:14:03.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.093013+0000", 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000225687 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.098959+0000", 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059459059999999999 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123735+0000", 2024-09-16T11:14:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024775755999999999 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123753+0000", 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8065999999999999e-05 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123836+0000", 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-05 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123887+0000", 2024-09-16T11:14:03.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0559000000000002e-05 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop_reply(client.4533.0:15969 4.d e47/36 ondisk, result = 0)", 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.158015621000001, 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033797389999999997, 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089912+0000", 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089913+0000", 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.04e-06 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089921+0000", 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2309999999999996e-06 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089941+0000", 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0494000000000001e-05 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.089948+0000", 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4779999999999996e-06 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123470+0000", 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.033522226000000002 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123488+0000", 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8269000000000001e-05 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123709+0000", 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220652 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26656 4.9 4:95b731cd:::rbd_data.11b5499bda95.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.297027282999998, 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.054121295999999999, 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:03.267 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26656 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950902+0000", 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950914+0000", 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1724e-05 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950943+0000", 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9621e-05 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950956+0000", 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.238e-05 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951090+0000", 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013472399999999999 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951475+0000", 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000384378 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.964699+0000", 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013224217999999999 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004806+0000", 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.040106941 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004820+0000", 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3909e-05 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004895+0000", 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4905000000000002e-05 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.005021+0000", 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012639300000000001 2024-09-16T11:14:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49320 4.1a 4:5c7f3453:::rbd_data.11b5eaebe1b2.0000000000000004:11 [read 1466368~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134586+0000", 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113340757, 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029924813000000001, 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.273 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49320 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134586+0000", 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134586+0000", 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134587+0000", 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4499999999999998e-07 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134588+0000", 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2500000000000001e-06 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134591+0000", 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134593+0000", 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1720000000000001e-06 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164197+0000", 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029604001000000001 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164256+0000", 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8882000000000003e-05 2024-09-16T11:14:03.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164511+0000", 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00025535 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49321 4.1a 4:5c7f3453:::rbd_data.11b5eaebe1b2.0000000000000004:11 [read 1531904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113332266, 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030292619999999999, 2024-09-16T11:14:03.277 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49321 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134595+0000", 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134596+0000", 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1480000000000001e-06 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134599+0000", 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.982e-06 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134601+0000", 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9970000000000001e-06 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164533+0000", 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029932041 2024-09-16T11:14:03.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164600+0000", 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6744999999999999e-05 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164888+0000", 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028745099999999998 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.281 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49361 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1007616~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134747+0000", 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113179909999999, 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029995797000000001, 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49361 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134747+0000", 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134747+0000", 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134748+0000", 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.67e-07 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134749+0000", 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.331e-06 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134752+0000", 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2830000000000002e-06 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134755+0000", 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:03.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164180+0000", 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029425617000000001 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164403+0000", 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022296100000000001 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164743+0000", 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00033982000000000001 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49362 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1073152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.11317051, 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030314636999999998, 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49362 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134757+0000", 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4999999999999999e-07 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134758+0000", 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2929999999999999e-06 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134761+0000", 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134763+0000", 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.12e-06 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164759+0000", 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029996030999999999 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.164824+0000", 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4828999999999999e-05 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165071+0000", 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024724099999999999 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.290 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49363 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1138688~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113160539000001, 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030709633, 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49363 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134767+0000", 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5400000000000002e-07 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134768+0000", 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1880000000000001e-06 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134771+0000", 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.106e-06 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134774+0000", 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4339999999999999e-06 2024-09-16T11:14:03.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165113+0000", 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030339123999999999 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165176+0000", 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3652999999999995e-05 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165476+0000", 2024-09-16T11:14:03.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000299874 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49364 4.e 4:72dd2813:::rbd_data.11b5eaebe1b2.0000000000000007:11 [read 1204224~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113151594, 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.03118348, 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.295 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49364 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134776+0000", 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4699999999999998e-07 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134777+0000", 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134780+0000", 2024-09-16T11:14:03.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8040000000000002e-06 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134782+0000", 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9269999999999999e-06 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165521+0000", 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030739417000000002 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165610+0000", 2024-09-16T11:14:03.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9164000000000002e-05 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165959+0000", 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034861899999999999 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.299 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49365 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113143366999999, 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.03122434, 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49365 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134784+0000", 2024-09-16T11:14:03.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134785+0000", 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.17e-06 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134788+0000", 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8049999999999998e-06 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134790+0000", 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.193e-06 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166000+0000", 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031209496999999999 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166008+0000", 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4239999999999993e-06 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49366 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113135100999999, 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031247255000000002, 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.304 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49366 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134792+0000", 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9799999999999999e-07 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134794+0000", 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2500000000000001e-06 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134797+0000", 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9359999999999999e-06 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134799+0000", 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166029+0000", 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031230734999999999 2024-09-16T11:14:03.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166039+0000", 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8959999999999998e-06 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49367 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113126615000001, 2024-09-16T11:14:03.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0312546, 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49367 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134801+0000", 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134802+0000", 2024-09-16T11:14:03.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1820000000000001e-06 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134805+0000", 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6960000000000001e-06 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134807+0000", 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.844e-06 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166051+0000", 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031244416000000001 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166055+0000", 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9160000000000003e-06 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49368 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.312 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113118603, 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031285238, 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49368 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:14:03.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134809+0000", 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134810+0000", 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1629999999999999e-06 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134813+0000", 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.92e-06 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134815+0000", 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0609999999999998e-06 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166084+0000", 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031268351 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166094+0000", 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.022e-05 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49369 4.e 4.c814bb4e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.113110189, 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031292820999999998, 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49369 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134817+0000", 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134818+0000", 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-06 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134821+0000", 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.134824+0000", 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:03.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166105+0000", 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.031281010999999997 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.166110+0000", 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2730000000000002e-06 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49460 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 679936~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135238+0000", 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.112688868999999, 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029937043999999999, 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49460 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135238+0000", 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135238+0000", 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135239+0000", 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135241+0000", 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.373e-06 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135247+0000", 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8259999999999998e-06 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135253+0000", 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7520000000000002e-06 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165064+0000", 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029811212 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165114+0000", 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.939e-05 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1870000000000002e-05 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49463 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 876544~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135272+0000", 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.112654769000001, 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030058985999999999, 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49463 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135272+0000", 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135272+0000", 2024-09-16T11:14:03.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135273+0000", 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.67e-07 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135274+0000", 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6479999999999999e-06 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135300+0000", 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5609000000000002e-05 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135323+0000", 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2892999999999999e-05 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165194+0000", 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029871051999999999 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165249+0000", 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5052000000000001e-05 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165331+0000", 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2465000000000004e-05 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:49466 4.1a 4:59977a19:::rbd_data.11b5caf806a8.0000000000000008:10 [read 1073152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.329 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.135329+0000", 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.112598119999999, 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030139170999999999, 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49466 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135329+0000", 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135329+0000", 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135330+0000", 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0200000000000002e-07 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135333+0000", 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.461e-06 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135340+0000", 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4980000000000003e-06 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.135346+0000", 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2160000000000003e-06 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165362+0000", 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.030016571999999998 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165402+0000", 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9749999999999997e-05 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.165468+0000", 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6072e-05 2024-09-16T11:14:03.333 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:65752 4.9 4:9275d7b8:::rbd_data.11b5a972b677.0000000000000004:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:57.791153+0000", 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.4567746850000001, 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.036160642, 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.334 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 65752 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791153+0000", 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791153+0000", 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791155+0000", 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791165+0000", 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0265000000000001e-05 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791208+0000", 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3439999999999997e-05 2024-09-16T11:14:03.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791220+0000", 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1489e-05 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791272+0000", 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2280000000000001e-05 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.791334+0000", 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1858000000000003e-05 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.798153+0000", 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0068187930000000001 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.827215+0000", 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.029061917999999999 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.827228+0000", 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3298e-05 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.827278+0000", 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9475999999999999e-05 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:57.827313+0000", 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5593000000000003e-05 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.339 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:65823 4.9 4:976608cf:::rbd_data.11b5a972b677.0000000000000003:head [set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:58.035387+0000", 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.212540637, 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.035638734999999998, 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 65823 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.340 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035387+0000", 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035387+0000", 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035388+0000", 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8169999999999999e-06 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035398+0000", 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4369999999999996e-06 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035420+0000", 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2526000000000001e-05 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035435+0000", 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4148000000000001e-05 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035530+0000", 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5676999999999999e-05 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.035665+0000", 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135074 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.043987+0000", 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0083219520000000005 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.070844+0000", 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.026857208 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.070864+0000", 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9539999999999999e-05 2024-09-16T11:14:03.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.070960+0000", 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5588999999999996e-05 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:58.071025+0000", 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5766999999999996e-05 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.345 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.347 INFO:teuthology.orchestra.run.smithi162.stdout:2809 writesame 0x5d2d717 thru 0x5d2f621 data_size 0x3(0x1f0b bytes) 2024-09-16T11:14:03.347 INFO:teuthology.orchestra.run.smithi162.stdout:2810 write 0x84ba9af thru 0x84c199e (0x6ff0 bytes) 2024-09-16T11:14:03.347 INFO:teuthology.orchestra.run.smithi162.stdout:2811 read 0x5841743 thru 0x5843ac3 (0x2381 bytes) 2024-09-16T11:14:03.348 INFO:teuthology.orchestra.run.smithi162.stdout:2813 read 0x2d4041c thru 0x2d4baab (0xb690 bytes) 2024-09-16T11:14:03.348 INFO:teuthology.orchestra.run.smithi162.stdout:2814 trunc from 0x84c199f to 0x1298f7b 2024-09-16T11:14:03.348 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:03.340+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:03.348 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:03.379 INFO:teuthology.orchestra.run.smithi162.stdout:2815 writesame 0xdb37a3a thru 0xdb37f16 data_size 0x5(0x4dd bytes) 2024-09-16T11:14:03.379 INFO:teuthology.orchestra.run.smithi162.stdout:2816 read 0x46df6a3 thru 0x46e5bc7 (0x6525 bytes) 2024-09-16T11:14:03.379 INFO:teuthology.orchestra.run.smithi162.stdout:2817 write 0x21de51e thru 0x21eb030 (0xcb13 bytes) 2024-09-16T11:14:03.388 INFO:teuthology.orchestra.run.smithi162.stdout:2819 write 0x5592c2e thru 0x5594d40 (0x2113 bytes) 2024-09-16T11:14:03.395 INFO:teuthology.orchestra.run.smithi162.stdout:2820 writesame 0xb5b14a thru 0xb62892 data_size 0x51(0x7749 bytes) 2024-09-16T11:14:03.413 INFO:teuthology.orchestra.run.smithi162.stdout:2821 read 0x2b33f02 thru 0x2b41228 (0xd327 bytes) 2024-09-16T11:14:03.413 INFO:teuthology.orchestra.run.smithi162.stdout:2822 writesame 0x2ea6e84 thru 0x2ea8bf2 data_size 0xb(0x1d6f bytes) 2024-09-16T11:14:03.420 INFO:teuthology.orchestra.run.smithi162.stdout:2823 write 0xc956f97 thru 0xc962bb1 (0xbc1b bytes) 2024-09-16T11:14:03.428 INFO:teuthology.orchestra.run.smithi162.stdout:2824 punch from 0xd81880c to 0xd81fc57, (0x744b bytes) 2024-09-16T11:14:03.428 INFO:teuthology.orchestra.run.smithi162.stdout:2826 read 0x24691b6 thru 0x2474b53 (0xb99e bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2827 read 0x2dfa39a thru 0x2dfdb44 (0x37ab bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2828 read 0x9c09297 thru 0x9c0b1d3 (0x1f3d bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2829 read 0x9d15e76 thru 0x9d24b1e (0xeca9 bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2830 punch from 0x9ad7f1d to 0x9adfaf9, (0x7bdc bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2831 read 0x1707bfc thru 0x170c906 (0x4d0b bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2832 read 0x6e0e717 thru 0x6e1dfcf (0xf8b9 bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2834 punch from 0x7db9bf3 to 0x7dbfa7a, (0x5e87 bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2835 read 0x8deebbb thru 0x8df30d8 (0x451e bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2836 punch from 0x18815e9 to 0x188ea04, (0xd41b bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2838 read 0x8d9e7b5 thru 0x8da7a98 (0x92e4 bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2839 read 0xc02f46c thru 0xc033f13 (0x4aa8 bytes) 2024-09-16T11:14:03.429 INFO:teuthology.orchestra.run.smithi162.stdout:2840 write 0xbc35bba thru 0xbc3c660 (0x6aa7 bytes) 2024-09-16T11:14:03.434 INFO:teuthology.orchestra.run.smithi162.stdout:2841 writesame 0xcbb0476 thru 0xcbbe01f data_size 0x1f(0xdbaa bytes) 2024-09-16T11:14:03.448 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:03.440+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:03.454 INFO:teuthology.orchestra.run.smithi162.stdout:2842 write 0xe8e7d50 thru 0xe8ec0d5 (0x4386 bytes) 2024-09-16T11:14:03.455 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.456 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:03.456 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:03.456 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.469 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:03.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:14:03.471 INFO:teuthology.orchestra.run.smithi162.stdout:2843 trunc from 0xe8ec0d6 to 0xad40ece 2024-09-16T11:14:03.489 INFO:teuthology.orchestra.run.smithi162.stdout:2844 trunc from 0xad40ece to 0xa62823b 2024-09-16T11:14:03.508 INFO:teuthology.orchestra.run.smithi162.stdout:2845 writesame 0xb41fbe6 thru 0xb42f15a data_size 0x1(0xf575 bytes) 2024-09-16T11:14:03.509 INFO:teuthology.orchestra.run.smithi162.stdout:2846 read 0x19815d thru 0x19d07a (0x4f1e bytes) 2024-09-16T11:14:03.511 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:03.511 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:03.511 INFO:teuthology.orchestra.run.smithi162.stdout:2850 read 0x34f5636 thru 0x3504585 (0xef50 bytes) 2024-09-16T11:14:03.545 INFO:teuthology.orchestra.run.smithi162.stdout:2852 write 0xd587c4a thru 0xd58d768 (0x5b1f bytes) 2024-09-16T11:14:03.548 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:03.540+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:03.551 INFO:teuthology.orchestra.run.smithi162.stdout:2853 read 0x23d9f6f thru 0x23e771e (0xd7b0 bytes) 2024-09-16T11:14:03.551 INFO:teuthology.orchestra.run.smithi162.stdout:2854 read 0x612d83f thru 0x613d152 (0xf914 bytes) 2024-09-16T11:14:03.551 INFO:teuthology.orchestra.run.smithi162.stdout:2855 write 0xa7f24cf thru 0xa8005d4 (0xe106 bytes) 2024-09-16T11:14:03.566 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.566 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:03.566 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:03.566 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:03.566 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.568 INFO:teuthology.orchestra.run.smithi162.stdout:2856 trunc from 0xd58d769 to 0x7db873e 2024-09-16T11:14:03.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:03.583 INFO:teuthology.orchestra.run.smithi162.stdout:2858 read 0x3c571c6 thru 0x3c61085 (0x9ec0 bytes) 2024-09-16T11:14:03.588 INFO:teuthology.orchestra.run.smithi162.stdout:2860 trunc from 0x7db873e to 0x59b7df1 2024-09-16T11:14:03.597 INFO:teuthology.orchestra.run.smithi162.stdout:2861 clone 22 order 19 su 65536 sc 7 2024-09-16T11:14:03.649 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:03.644+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.805598918000001, 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:03.673 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:03.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:03.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.677 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637627268999999, 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.678 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:03.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637615708, 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:03.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637602506, 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:03.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637590066, 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.691 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.695 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637576555000001, 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.700 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.63756564, 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:03.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637539722, 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:03.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:03.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637524268, 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:03.710 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:03.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637514381999999, 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:03.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:03.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637504619, 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.720 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:03.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:03.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:03.724 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637494441999999, 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:03.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.63747796, 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.729 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:03.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.733 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637463263000001, 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:03.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:03.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.738 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637444820000001, 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:03.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:03.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637427871, 2024-09-16T11:14:03.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.748 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637408658, 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:03.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637391442, 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:03.753 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:03.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:03.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.637370188, 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:03.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.762 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.63735022, 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:03.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:03.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:03.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:03.767 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:03.769 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:03.760+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:03.870 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:03.861+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:03.876 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.876 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:03.877 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:03.877 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.885 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:03.971 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:03.965+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:03.989 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:03.989 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:03.989 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:03.989 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:03.989 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:03.998 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:04.072 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:04.064+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:04.100 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.910553776, 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:04.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.906195249, 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50301 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.902576367, 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11841462, 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50301 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:04.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193762+0000", 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7399999999999998e-07 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193764+0000", 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193771+0000", 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193775+0000", 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3619999999999999e-06 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311756+0000", 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11798077999999999 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311868+0000", 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112605 2024-09-16T11:14:04.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312176+0000", 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030757000000000001 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.902556719, 2024-09-16T11:14:04.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:04.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:04.120 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.902537933, 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:04.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:04.124 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.90252061, 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.125 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:04.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.130 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.902503141, 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.131 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:04.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898662681999999, 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.139 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898652566999999, 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:04.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:04.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898642686000001, 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.145 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:04.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:04.149 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.89863351, 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:04.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:04.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898617442999999, 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:04.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:04.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898601048, 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:04.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898591474, 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.163 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:04.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.166 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.898581938, 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.167 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:04.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50319 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.894535544, 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118443188, 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50319 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:04.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201804+0000", 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-06 2024-09-16T11:14:04.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201807+0000", 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201809+0000", 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320222+0000", 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118412609 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320245+0000", 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3805999999999999e-05 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.894525281, 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:04.175 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:04.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:04.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.894515899, 2024-09-16T11:14:04.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:04.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.894507240999999, 2024-09-16T11:14:04.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:04.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.894498006999999, 2024-09-16T11:14:04.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:04.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:04.191 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:04.192 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:04.193 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:04.194 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:04.188+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:04.294 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:04.288+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:04.294 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:04.295 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:04.295 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:04.295 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:04.304 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:04.337 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:04.356 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:04.375 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:04.394 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:04.394 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:04.388+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:04.399 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:04.408 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:04.415 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:04.435 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:04.456 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:04.477 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:04.479 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:04.479 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:04.479 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:04.479 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:04.480 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:04.480 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:04.480 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:04.480 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:04.483 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:14:04.495 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:04.489+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:04.514 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:04.514 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:04.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:04.596 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:04.589+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:04.598 INFO:teuthology.orchestra.run.smithi162.stdout:leaving image image_client.0-clone21 intact 2024-09-16T11:14:04.623 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:04.623 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:04.633 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:04.697 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:04.689+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:04.736 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:04.736 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:04.746 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:04.751 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-16T11:14:04.798 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:04.792+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:04.857 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:04.857 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:04.869 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:04.869 INFO:teuthology.orchestra.run.smithi162.stdout:2863 read 0x2ba44df thru 0x2baf25b (0xad7d bytes) 2024-09-16T11:14:04.878 INFO:teuthology.orchestra.run.smithi162.stdout:2864 read 0x324c741 thru 0x3250937 (0x41f7 bytes) 2024-09-16T11:14:04.899 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:04.892+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:04.904 INFO:teuthology.orchestra.run.smithi162.stdout:2865 write 0xe2485a3 thru 0xe2492cd (0xd2b bytes) 2024-09-16T11:14:04.909 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:04.909 INFO:teuthology.orchestra.run.smithi162.stdout:2868 writesame 0x317736f thru 0x317907b data_size 0x25(0x1d0d bytes) 2024-09-16T11:14:04.913 INFO:teuthology.orchestra.run.smithi162.stdout:2871 writesame 0x9bbc2b1 thru 0x9bca86b data_size 0x1(0xe5bb bytes) 2024-09-16T11:14:04.950 INFO:teuthology.orchestra.run.smithi162.stdout:2872 punch from 0x3357058 to 0x33596f5, (0x269d bytes) 2024-09-16T11:14:04.950 INFO:teuthology.orchestra.run.smithi162.stdout:2873 read 0x98c499e thru 0x98ca101 (0x5764 bytes) 2024-09-16T11:14:04.951 INFO:teuthology.orchestra.run.smithi162.stdout:2874 read 0xe1d32dd thru 0xe1e13db (0xe0ff bytes) 2024-09-16T11:14:04.972 INFO:teuthology.orchestra.run.smithi162.stdout:2877 trunc from 0xe2492ce to 0x13a49f0 2024-09-16T11:14:04.973 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:04.973 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:04.984 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:05.001 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:04.993+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:05.013 INFO:teuthology.orchestra.run.smithi162.stdout:2878 read 0x11aeb5f thru 0x11b8872 (0x9d14 bytes) 2024-09-16T11:14:05.015 INFO:teuthology.orchestra.run.smithi162.stdout:2880 trunc from 0x13a49f0 to 0x18a9a20 2024-09-16T11:14:05.026 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.051 INFO:teuthology.orchestra.run.smithi162.stdout:2882 writesame 0x83bf940 thru 0x83ca4c1 data_size 0x2(0xab82 bytes) 2024-09-16T11:14:05.066 INFO:teuthology.orchestra.run.smithi162.stdout:2883 writesame 0x9a9555b thru 0x9aa239f data_size 0x3b(0xce45 bytes) 2024-09-16T11:14:05.066 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.066 INFO:teuthology.orchestra.run.smithi162.stdout:2885 write 0x1fc0e1 thru 0x2092fb (0xd21b bytes) 2024-09-16T11:14:05.086 INFO:teuthology.orchestra.run.smithi162.stdout:2886 read 0x2af7a0b thru 0x2afcd1a (0x5310 bytes) 2024-09-16T11:14:05.086 INFO:teuthology.orchestra.run.smithi162.stdout:2888 write 0x829ef13 thru 0x82adabe (0xebac bytes) 2024-09-16T11:14:05.101 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:05.096+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:05.109 INFO:teuthology.orchestra.run.smithi162.stdout:2889 write 0xccabd0a thru 0xccbb067 (0xf35e bytes) 2024-09-16T11:14:05.118 INFO:teuthology.orchestra.run.smithi162.stdout:2890 read 0xf3873c thru 0xf3e4da (0x5d9f bytes) 2024-09-16T11:14:05.120 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.120 INFO:teuthology.orchestra.run.smithi162.stdout:2892 punch from 0x6f866e0 to 0x6f9566c, (0xef8c bytes) 2024-09-16T11:14:05.122 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.122 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.124 INFO:teuthology.orchestra.run.smithi162.stdout:2893 trunc from 0xccbb068 to 0xadc5580 2024-09-16T11:14:05.137 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:05.149 INFO:teuthology.orchestra.run.smithi162.stdout:2894 punch from 0x482b70c to 0x4837402, (0xbcf6 bytes) 2024-09-16T11:14:05.149 INFO:teuthology.orchestra.run.smithi162.stdout:2896 read 0x6214016 thru 0x6214022 (0xd bytes) 2024-09-16T11:14:05.150 INFO:teuthology.orchestra.run.smithi162.stdout:2897 flatten 2024-09-16T11:14:05.202 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:05.197+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:05.218 INFO:teuthology.orchestra.run.smithi162.stdout:2898 writesame 0x8cc7bb9 thru 0x8cd248e data_size 0x1(0xa8d6 bytes) 2024-09-16T11:14:05.246 INFO:teuthology.orchestra.run.smithi162.stdout:2900 read 0x4cadc4a thru 0x4cbab5e (0xcf15 bytes) 2024-09-16T11:14:05.247 INFO:teuthology.orchestra.run.smithi162.stdout:2901 read 0x84e4278 thru 0x84eabfe (0x6987 bytes) 2024-09-16T11:14:05.247 INFO:teuthology.orchestra.run.smithi162.stdout:2902 read 0x9307403 thru 0x93096e1 (0x22df bytes) 2024-09-16T11:14:05.247 INFO:teuthology.orchestra.run.smithi162.stdout:2903 read 0x61606e5 thru 0x61686d0 (0x7fec bytes) 2024-09-16T11:14:05.247 INFO:teuthology.orchestra.run.smithi162.stdout:2904 read 0x1c3a5d8 thru 0x1c457aa (0xb1d3 bytes) 2024-09-16T11:14:05.247 INFO:teuthology.orchestra.run.smithi162.stdout:2905 writesame 0x3c0ddb0 thru 0x3c1a48a data_size 0x47(0xc6db bytes) 2024-09-16T11:14:05.255 INFO:teuthology.orchestra.run.smithi162.stdout:2906 trunc from 0xadc5580 to 0xc208fe1 2024-09-16T11:14:05.260 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.261 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.274 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:05.282 INFO:teuthology.orchestra.run.smithi162.stdout:2907 write 0xe3048b1 thru 0xe309be3 (0x5333 bytes) 2024-09-16T11:14:05.289 INFO:teuthology.orchestra.run.smithi162.stdout:2908 read 0x5550d4d thru 0x5555426 (0x46da bytes) 2024-09-16T11:14:05.289 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.289 INFO:teuthology.orchestra.run.smithi162.stdout:2910 writesame 0x8edb27 thru 0x8f6806 data_size 0x62(0x8ce0 bytes) 2024-09-16T11:14:05.297 INFO:teuthology.orchestra.run.smithi162.stdout:2911 punch from 0x3b84a75 to 0x3b92d58, (0xe2e3 bytes) 2024-09-16T11:14:05.298 INFO:teuthology.orchestra.run.smithi162.stdout:2912 read 0xac2d685 thru 0xac3a650 (0xcfcc bytes) 2024-09-16T11:14:05.298 INFO:teuthology.orchestra.run.smithi162.stdout:2913 write 0x9a88e64 thru 0x9a8b70c (0x28a9 bytes) 2024-09-16T11:14:05.303 INFO:teuthology.orchestra.run.smithi162.stdout:2914 writesame 0xb0c567a thru 0xb0cfc26 data_size 0x49(0xa5ad bytes) 2024-09-16T11:14:05.304 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:05.296+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:05.310 INFO:teuthology.orchestra.run.smithi162.stdout:2915 read 0xc0b94c thru 0xc0f7b8 (0x3e6d bytes) 2024-09-16T11:14:05.310 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.322 INFO:teuthology.orchestra.run.smithi162.stdout:2918 trunc from 0xe309be4 to 0x7bbce06 2024-09-16T11:14:05.346 INFO:teuthology.orchestra.run.smithi162.stdout:2919 punch from 0x1ccd0f6 to 0x1cd7fbc, (0xaec6 bytes) 2024-09-16T11:14:05.346 INFO:teuthology.orchestra.run.smithi162.stdout:2924 read 0x26111b1 thru 0x261e455 (0xd2a5 bytes) 2024-09-16T11:14:05.346 INFO:teuthology.orchestra.run.smithi162.stdout:2925 read 0x1e057fd thru 0x1e131f3 (0xd9f7 bytes) 2024-09-16T11:14:05.350 INFO:teuthology.orchestra.run.smithi162.stdout:2926 trunc from 0x7bbce06 to 0x560e2f7 2024-09-16T11:14:05.374 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.374 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.377 INFO:teuthology.orchestra.run.smithi162.stdout:2929 write 0x5fa303d thru 0x5fb0430 (0xd3f4 bytes) 2024-09-16T11:14:05.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:05.387 INFO:teuthology.orchestra.run.smithi162.stdout:2930 read 0xb06e6e thru 0xb13738 (0xc8cb bytes) 2024-09-16T11:14:05.401 INFO:teuthology.orchestra.run.smithi162.stdout:2932 write 0x74a7756 thru 0x74b5f57 (0xe802 bytes) 2024-09-16T11:14:05.404 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:05.397+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:05.410 INFO:teuthology.orchestra.run.smithi162.stdout:2933 read 0x6fa981f thru 0x6fb319d (0x997f bytes) 2024-09-16T11:14:05.410 INFO:teuthology.orchestra.run.smithi162.stdout:2934 read 0xf72eb6 thru 0xf7c536 (0x9681 bytes) 2024-09-16T11:14:05.410 INFO:teuthology.orchestra.run.smithi162.stdout:2935 punch from 0x5d494ac to 0x5d57cd6, (0xe82a bytes) 2024-09-16T11:14:05.410 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.410 INFO:teuthology.orchestra.run.smithi162.stdout:2937 read 0x5e7d8fb thru 0x5e88942 (0xb048 bytes) 2024-09-16T11:14:05.410 INFO:teuthology.orchestra.run.smithi162.stdout:2938 punch from 0x2102d21 to 0x2107ab2, (0x4d91 bytes) 2024-09-16T11:14:05.411 INFO:teuthology.orchestra.run.smithi162.stdout:2939 read 0x39d907 thru 0x3ad74d (0xfe47 bytes) 2024-09-16T11:14:05.433 INFO:teuthology.orchestra.run.smithi162.stdout:2941 write 0xd55665b thru 0xd558d11 (0x26b7 bytes) 2024-09-16T11:14:05.440 INFO:teuthology.orchestra.run.smithi162.stdout:2942 read 0x75cc9ec thru 0x75d3bbe (0x71d3 bytes) 2024-09-16T11:14:05.453 INFO:teuthology.orchestra.run.smithi162.stdout:2944 writesame 0xdb867bb thru 0xdb9585e data_size 0x2(0xf0a4 bytes) 2024-09-16T11:14:05.453 INFO:teuthology.orchestra.run.smithi162.stdout:2945 writesame 0x147a505 thru 0x1488dda data_size 0x1(0xe8d6 bytes) 2024-09-16T11:14:05.485 INFO:teuthology.orchestra.run.smithi162.stdout:2947 punch from 0x54d21c to 0x558848, (0xb62c bytes) 2024-09-16T11:14:05.485 INFO:teuthology.orchestra.run.smithi162.stdout:2948 write 0x1b2624 thru 0x1bc0f6 (0x9ad3 bytes) 2024-09-16T11:14:05.488 INFO:teuthology.orchestra.run.smithi162.stdout:2950 read 0x2dbd33b thru 0x2dc2bb6 (0x587c bytes) 2024-09-16T11:14:05.488 INFO:teuthology.orchestra.run.smithi162.stdout:2951 read 0xabfa314 thru 0xabfda14 (0x3701 bytes) 2024-09-16T11:14:05.488 INFO:teuthology.orchestra.run.smithi162.stdout:2953 write 0x49ad9e3 thru 0x49b43ca (0x69e8 bytes) 2024-09-16T11:14:05.489 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.489 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.496 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:05.508 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:05.500+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:05.516 INFO:teuthology.orchestra.run.smithi162.stdout:2956 trunc from 0xdb9585f to 0x1277b4e 2024-09-16T11:14:05.537 INFO:teuthology.orchestra.run.smithi162.stdout:2957 read 0x1051647 thru 0x105e8df (0xd299 bytes) 2024-09-16T11:14:05.537 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.549 INFO:teuthology.orchestra.run.smithi162.stdout:2962 write 0x26aeec0 thru 0x26b6616 (0x7757 bytes) 2024-09-16T11:14:05.556 INFO:teuthology.orchestra.run.smithi162.stdout:2963 read 0x1b85b3c thru 0x1b8cf22 (0x73e7 bytes) 2024-09-16T11:14:05.580 INFO:teuthology.orchestra.run.smithi162.stdout:2964 write 0x8ad870f thru 0x8ae6434 (0xdd26 bytes) 2024-09-16T11:14:05.589 INFO:teuthology.orchestra.run.smithi162.stdout:2966 read 0x759deb6 thru 0x75a518a (0x72d5 bytes) 2024-09-16T11:14:05.589 INFO:teuthology.orchestra.run.smithi162.stdout:2967 read 0x24733db thru 0x247749c (0x40c2 bytes) 2024-09-16T11:14:05.589 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.589 INFO:teuthology.orchestra.run.smithi162.stdout:2970 read 0x48d99fd thru 0x48e0474 (0x6a78 bytes) 2024-09-16T11:14:05.589 INFO:teuthology.orchestra.run.smithi162.stdout:2972 read 0x343380e thru 0x3436060 (0x2853 bytes) 2024-09-16T11:14:05.589 INFO:teuthology.orchestra.run.smithi162.stdout:2973 write 0x61f8b7d thru 0x6204228 (0xb6ac bytes) 2024-09-16T11:14:05.596 INFO:teuthology.orchestra.run.smithi162.stdout:2974 write 0x2513cc0 thru 0x251ccc3 (0x9004 bytes) 2024-09-16T11:14:05.606 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:05.600+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:05.608 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.608 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.613 INFO:teuthology.orchestra.run.smithi162.stdout:2975 writesame 0x8ff5579 thru 0x8ffb3ac data_size 0x2(0x5e34 bytes) 2024-09-16T11:14:05.613 INFO:teuthology.orchestra.run.smithi162.stdout:2977 trunc from 0x8ffb3ad to 0x8f3257b 2024-09-16T11:14:05.617 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:05.625 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.625 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:05.625 INFO:teuthology.orchestra.run.smithi162.stdout:2980 punch from 0x5d2be98 to 0x5d31704, (0x586c bytes) 2024-09-16T11:14:05.636 INFO:teuthology.orchestra.run.smithi162.stdout:2981 writesame 0x9542813 thru 0x954b74e data_size 0x4(0x8f3c bytes) 2024-09-16T11:14:05.636 INFO:teuthology.orchestra.run.smithi162.stdout:2982 read 0x2ee1026 thru 0x2eedf88 (0xcf63 bytes) 2024-09-16T11:14:05.636 INFO:teuthology.orchestra.run.smithi162.stdout:2984 read 0x71350a9 thru 0x7137df6 (0x2d4e bytes) 2024-09-16T11:14:05.644 INFO:teuthology.orchestra.run.smithi162.stdout:2985 trunc from 0x954b74f to 0x5afc8c9 2024-09-16T11:14:05.664 INFO:teuthology.orchestra.run.smithi162.stdout:2986 writesame 0xe5d47c thru 0xe5e028 data_size 0x7(0xbad bytes) 2024-09-16T11:14:05.683 INFO:teuthology.orchestra.run.smithi162.stdout:2987 write 0x7652bb6 thru 0x765e8d3 (0xbd1e bytes) 2024-09-16T11:14:05.693 INFO:teuthology.orchestra.run.smithi162.stdout:2988 trunc from 0x765e8d4 to 0x93014b1 2024-09-16T11:14:05.704 INFO:teuthology.orchestra.run.smithi162.stdout:2990 writesame 0x7ec1237 thru 0x7ecc7a8 data_size 0x19(0xb572 bytes) 2024-09-16T11:14:05.707 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:05.700+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:05.721 INFO:teuthology.orchestra.run.smithi162.stdout:2991 trunc from 0x93014b1 to 0xd58f201 2024-09-16T11:14:05.729 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.729 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.733 INFO:teuthology.orchestra.run.smithi162.stdout:2992 read 0x53c97a2 thru 0x53c9843 (0xa2 bytes) 2024-09-16T11:14:05.733 INFO:teuthology.orchestra.run.smithi162.stdout:2993 writesame 0x9fb0801 thru 0x9fb7d1c data_size 0xa(0x751c bytes) 2024-09-16T11:14:05.739 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:05.742 INFO:teuthology.orchestra.run.smithi162.stdout:2995 writesame 0x9ea1d32 thru 0x9eaf7fa data_size 0x1(0xdac9 bytes) 2024-09-16T11:14:05.790 INFO:teuthology.orchestra.run.smithi162.stdout:2996 write 0x5d5b2af thru 0x5d662c9 (0xb01b bytes) 2024-09-16T11:14:05.798 INFO:teuthology.orchestra.run.smithi162.stdout:2997 punch from 0x692b0f6 to 0x693ab9c, (0xfaa6 bytes) 2024-09-16T11:14:05.798 INFO:teuthology.orchestra.run.smithi162.stdout:2998 read 0xa355a8d thru 0xa35d871 (0x7de5 bytes) 2024-09-16T11:14:05.798 INFO:teuthology.orchestra.run.smithi162.stdout:3000 clone 23 order 21 su 16384 sc 13 2024-09-16T11:14:05.807 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:05.800+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:05.842 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.842 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.851 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:05.909 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:05.905+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:05.948 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:05.948 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:05.957 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:06.010 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:06.005+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:06.065 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:06.066 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:06.075 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:06.111 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:06.104+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:06.179 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:06.180 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:06.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:06.212 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:06.204+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:06.287 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:06.287 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:06.296 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:06.313 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:06.309+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:06.394 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:06.394 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:06.403 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:06.413 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:06.409+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:06.495 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:06.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:14:06.508 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:06.508 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:06.514 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:06.509+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:06.518 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:06.615 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:06.613+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:06.634 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:06.655 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:06.667 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:06.667 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:06.675 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:06.676 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:06.695 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:06.715 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:06.716 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:06.708+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:06.736 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:06.759 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:06.779 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:06.781 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:06.782 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-16T11:14:06.783 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:06.783 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:06.783 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:06.783 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:06.792 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:06.818 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:06.813+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:06.915 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:06.915 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:06.915 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:06.915 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:06.915 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:06.919 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:06.917+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:06.924 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:06.931 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:06.931 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:06.931 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T11:14:06.932 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.694036 2024-09-16T11:14:06.932 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.6940357569072861 2024-09-16T11:14:07.020 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:07.012+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:07.027 INFO:teuthology.orchestra.run.smithi162.stdout:3001 write 0xaea1a8 thru 0xaeccdc (0x2b35 bytes) 2024-09-16T11:14:07.049 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.057 INFO:teuthology.orchestra.run.smithi162.stdout:3003 trunc from 0xd58f201 to 0x7eb6087 2024-09-16T11:14:07.081 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:07.082 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.892819805999999, 2024-09-16T11:14:07.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:07.083 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.083 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.083 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.083 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.083 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:07.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.888461279000001, 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50301 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.884842397, 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11841462, 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50301 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193762+0000", 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7399999999999998e-07 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193764+0000", 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193771+0000", 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193775+0000", 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3619999999999999e-06 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311756+0000", 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11798077999999999 2024-09-16T11:14:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311868+0000", 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112605 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312176+0000", 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030757000000000001 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.884822749000001, 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.884803963, 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:07.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:07.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.108 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.108 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:07.108 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.884786640000002, 2024-09-16T11:14:07.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:07.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:07.113 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.884769170999999, 2024-09-16T11:14:07.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:07.114 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.114 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:07.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.118 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880928711999999, 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880918597000001, 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:07.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:07.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.127 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880908716, 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:07.131 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880899540000001, 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:07.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880883473000001, 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:07.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:07.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:07.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880867078000001, 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:07.142 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:07.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880857504000002, 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:07.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:07.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:07.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:07.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.880847968000001, 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:07.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:07.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50319 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.876801574000002, 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118443188, 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50319 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201804+0000", 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-06 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201807+0000", 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201809+0000", 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320222+0000", 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118412609 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320245+0000", 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3805999999999999e-05 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.876791311000002, 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.162 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:07.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.876781929, 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:07.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:07.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.876773271000001, 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.173 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.876764037000001, 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.178 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.178 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:07.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:07.180 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:07.177+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3004 read 0x227439e thru 0x228042b (0xc08e bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3005 read 0x5e93fc4 thru 0x5ea3c15 (0xfc52 bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3006 trunc from 0x7eb6087 to 0x3da0b95 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3009 punch from 0x28ff72c to 0x29067b6, (0x708a bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3011 read 0x5c4530 thru 0x5c7c3d (0x370e bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3012 write 0x56a87f thru 0x57646c (0xbbee bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3013 punch from 0xdd95ba to 0xddb62d, (0x2073 bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3014 punch from 0x2f96cb4 to 0x2fa0a1f, (0x9d6b bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:3016 writesame 0x2b0cf5c thru 0x2b1a8f7 data_size 0x13(0xd99c bytes) 2024-09-16T11:14:07.181 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.206 INFO:teuthology.orchestra.run.smithi162.stdout:3019 write 0xe97a64b thru 0xe987938 (0xd2ee bytes) 2024-09-16T11:14:07.217 INFO:teuthology.orchestra.run.smithi162.stdout:3020 read 0x52b0c40 thru 0x52b9ff5 (0x93b6 bytes) 2024-09-16T11:14:07.217 INFO:teuthology.orchestra.run.smithi162.stdout:3021 read 0x464912e thru 0x465562e (0xc501 bytes) 2024-09-16T11:14:07.217 INFO:teuthology.orchestra.run.smithi162.stdout:3022 writesame 0xe745e4e thru 0xe752433 data_size 0x2(0xc5e6 bytes) 2024-09-16T11:14:07.247 INFO:teuthology.orchestra.run.smithi162.stdout:3024 trunc from 0xe987939 to 0x42105c1 2024-09-16T11:14:07.268 INFO:teuthology.orchestra.run.smithi162.stdout:3025 read 0x1ee947e thru 0x1ef25d7 (0x915a bytes) 2024-09-16T11:14:07.269 INFO:teuthology.orchestra.run.smithi162.stdout:3028 writesame 0x1767115 thru 0x1772852 data_size 0x5(0xb73e bytes) 2024-09-16T11:14:07.280 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:07.277+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:07.293 INFO:teuthology.orchestra.run.smithi162.stdout:3029 read 0x382afbc thru 0x38319a5 (0x69ea bytes) 2024-09-16T11:14:07.294 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:07.294 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:07.294 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:07.294 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:07.294 INFO:teuthology.orchestra.run.smithi162.stdout:3030 read 0x222f8fd thru 0x2231630 (0x1d34 bytes) 2024-09-16T11:14:07.295 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.303 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:07.304 INFO:teuthology.orchestra.run.smithi162.stdout:3032 writesame 0x5231600 thru 0x523e7b5 data_size 0x11(0xd1b6 bytes) 2024-09-16T11:14:07.304 INFO:teuthology.orchestra.run.smithi162.stdout:3036 read 0x1364f58 thru 0x1369a51 (0x4afa bytes) 2024-09-16T11:14:07.305 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.305 INFO:teuthology.orchestra.run.smithi162.stdout:3039 write 0x36f4ba8 thru 0x36f4ec1 (0x31a bytes) 2024-09-16T11:14:07.309 INFO:teuthology.orchestra.run.smithi162.stdout:3040 read 0x15b6bcf thru 0x15c5288 (0xe6ba bytes) 2024-09-16T11:14:07.311 INFO:teuthology.orchestra.run.smithi162.stdout:3041 read 0x1e17b6c thru 0x1e1c15a (0x45ef bytes) 2024-09-16T11:14:07.316 INFO:teuthology.orchestra.run.smithi162.stdout:3042 read 0x4d10ab0 thru 0x4d11686 (0xbd7 bytes) 2024-09-16T11:14:07.316 INFO:teuthology.orchestra.run.smithi162.stdout:3043 read 0x77f5aa thru 0x7828d9 (0x3330 bytes) 2024-09-16T11:14:07.316 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.316 INFO:teuthology.orchestra.run.smithi162.stdout:3046 writesame 0xedf174 thru 0xee8edd data_size 0x1(0x9d6a bytes) 2024-09-16T11:14:07.331 INFO:teuthology.orchestra.run.smithi162.stdout:3047 writesame 0x1e8c48 thru 0x1f695f data_size 0x64(0xdd18 bytes) 2024-09-16T11:14:07.336 INFO:teuthology.orchestra.run.smithi162.stdout:3049 write 0x2d99f18 thru 0x2da11c7 (0x72b0 bytes) 2024-09-16T11:14:07.347 INFO:teuthology.orchestra.run.smithi162.stdout:3051 read 0x1af2e8 thru 0x1b6906 (0x761f bytes) 2024-09-16T11:14:07.371 INFO:teuthology.orchestra.run.smithi162.stdout:3052 write 0xd814823 thru 0xd81ed85 (0xa563 bytes) 2024-09-16T11:14:07.381 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:07.377+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:07.397 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:07.397 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:07.397 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:07.397 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:07.397 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:07.403 INFO:teuthology.orchestra.run.smithi162.stdout:3053 trunc from 0xd81ed86 to 0x3b43e4 2024-09-16T11:14:07.406 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:07.432 INFO:teuthology.orchestra.run.smithi162.stdout:3056 read 0x1ce78f thru 0x1d95b9 (0xae2b bytes) 2024-09-16T11:14:07.433 INFO:teuthology.orchestra.run.smithi162.stdout:3057 punch from 0x152f0d to 0x161859, (0xe94c bytes) 2024-09-16T11:14:07.457 INFO:teuthology.orchestra.run.smithi162.stdout:3058 write 0x8cafd68 thru 0x8cb87c6 (0x8a5f bytes) 2024-09-16T11:14:07.467 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.467 INFO:teuthology.orchestra.run.smithi162.stdout:3060 writesame 0x11bc223 thru 0x11c306c data_size 0x13(0x6e4a bytes) 2024-09-16T11:14:07.471 INFO:teuthology.orchestra.run.smithi162.stdout:3061 read 0x1f9fd74 thru 0x1f9fe66 (0xf3 bytes) 2024-09-16T11:14:07.471 INFO:teuthology.orchestra.run.smithi162.stdout:3063 punch from 0x7a1816c to 0x7a25185, (0xd019 bytes) 2024-09-16T11:14:07.471 INFO:teuthology.orchestra.run.smithi162.stdout:3064 writesame 0x78841c2 thru 0x7891ab0 data_size 0x1d(0xd8ef bytes) 2024-09-16T11:14:07.482 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:07.482 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:07.476+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:07.497 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.1 primary-affinity to 0.694036 (8454842) 2024-09-16T11:14:07.497 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:14:07.504 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:07.504 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:07.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:07.504 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.721829815999996, 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:07.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:07.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:07.511 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333904115999999, 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.512 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:07.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:07.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.517 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:07.520 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333752786000005, 2024-09-16T11:14:07.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:07.520 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.520 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:07.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333701300000001, 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:07.526 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.532 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333631151999995, 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:07.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:07.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.538 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333458682, 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:07.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333400359999999, 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.544 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:07.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:07.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333248240000003, 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:07.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:07.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.555 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333069721000001, 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:07.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:07.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:07.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:07.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.333036747999998, 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:07.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:14:07.568 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.780871255999998, 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011997318, 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 71 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716978+0000", 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716979+0000", 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716983+0000", 2024-09-16T11:14:07.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716988+0000", 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.716992+0000", 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725000+0000", 2024-09-16T11:14:07.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080079580000000008 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725138+0000", 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013814 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.725348+0000", 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021024499999999999 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728729+0000", 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033810590000000001 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728802+0000", 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2144999999999995e-05 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728906+0000", 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010414800000000001 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.728976+0000", 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9987999999999995e-05 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.780849023000002, 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.574 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:07.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:07.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.780720226, 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:07.580 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:07.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.780661881, 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.586 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:07.591 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.780607492000001, 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:07.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:07.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.780473185000005, 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.598 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:07.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.999317816000001, 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:07.604 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:07.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:07.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.999230503999996, 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.610 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:07.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:07.616 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 75.998992376000004, 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:07.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:07.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.3186123670000001, 2024-09-16T11:14:07.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:07.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:07.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.626 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:07.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:07.627 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:07.629 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:07.630 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:07.620+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:07.729 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:07.725+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:07.781 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:07.781 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:07.781 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:07.781 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:07.790 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:07.830 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:07.824+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:07.894 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:07.894 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:07.894 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:07.894 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:07.894 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:07.903 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:07.949 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:07.944+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.830520126, 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:08.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.830442383000001, 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.014 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:08.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.018 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.828076368000001, 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:08.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:08.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.828056852, 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.023 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:08.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.027 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.826919348000001, 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:08.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:08.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:08.032 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.826903449, 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:08.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:08.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.826879195, 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:08.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.826863935999999, 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:08.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.826848617, 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:08.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:08.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824826213000001, 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:08.050 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:08.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824811714999999, 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:08.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:08.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824797382, 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:08.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:08.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824783324999999, 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:08.063 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:08.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824769469, 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:08.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:08.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824755593999999, 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:08.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:08.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:08.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824741827, 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:08.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824694317999999, 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:08.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:08.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.824408500000001, 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:08.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:08.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:08.089 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.731744683999999, 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:08.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.731595502000001, 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:08.095 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:08.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:08.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.101 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:08.102 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:08.103 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:08.101+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:08.203 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:08.196+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4.f93f245c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.71629370599999997, 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.71633775499999996, 2024-09-16T11:14:08.210 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:08.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:08.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:08.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:14:08.215 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:08.220 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:08.304 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:08.301+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:08.377 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:08.377 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:08.377 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:08.380 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:08.380 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:08.386 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:08.404 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:08.396+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:08.490 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:08.490 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50255 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.326524695, 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063800420999999996, 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50255 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159696+0000", 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159697+0000", 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2e-07 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159699+0000", 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.114e-06 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159705+0000", 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9970000000000004e-06 2024-09-16T11:14:08.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159709+0000", 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223159+0000", 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063450079000000006 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223206+0000", 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7148000000000003e-05 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223497+0000", 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000290434 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:08.495 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.326508110999999, 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:08.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.326473057000001, 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:14:08.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.326456622999999, 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:14:08.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.326440314999999, 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:08.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:14:08.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:08.513 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.326424749000001, 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:14:08.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:14:08.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.325690255000001, 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:14:08.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:14:08.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.325674227, 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:08.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:14:08.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.526 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.325664397000001, 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:14:08.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:14:08.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:14:08.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.322354201, 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:14:08.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.536 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.322330851, 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:14:08.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.322311550999999, 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.541 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:08.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:08.545 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.322294069000002, 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:14:08.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.322263490000001, 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.550 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:14:08.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.322141673000001, 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:08.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.558 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.321700502999999, 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:14:08.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:14:08.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.321069631, 2024-09-16T11:14:08.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:08.567 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.321041349000001, 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:14:08.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:14:08.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.321026120999999, 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:14:08.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:14:08.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.321013219000001, 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:08.576 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:08.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:14:08.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:14:08.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:08.580 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:08.582 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3066 write 0xe6c7e40 thru 0xe6cdcb4 (0x5e75 bytes) 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3067 write 0x9923e5e thru 0x992f940 (0xbae3 bytes) 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3068 write 0x511a174 thru 0x511f094 (0x4f21 bytes) 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3069 punch from 0x1a70159 to 0x1a72390, (0x2237 bytes) 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3070 read 0xdf89986 thru 0xdf8f8bf (0x5f3a bytes) 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3071 write 0x18e18d7 thru 0x18e8cf5 (0x741f bytes) 2024-09-16T11:14:08.583 INFO:teuthology.orchestra.run.smithi162.stdout:3072 writesame 0x8a4eba8 thru 0x8a5c7b7 data_size 0x38(0xdc10 bytes) 2024-09-16T11:14:08.584 INFO:teuthology.orchestra.run.smithi162.stdout:3075 trunc from 0xe6cdcb5 to 0x71da315 2024-09-16T11:14:08.584 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:08.576+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:08.627 INFO:teuthology.orchestra.run.smithi162.stdout:3076 write 0xe6d3b93 thru 0xe6d5d3e (0x21ac bytes) 2024-09-16T11:14:08.633 INFO:teuthology.orchestra.run.smithi162.stdout:3077 writesame 0x72d966a thru 0x72dc2e4 data_size 0x1(0x2c7b bytes) 2024-09-16T11:14:08.642 INFO:teuthology.orchestra.run.smithi162.stdout:3078 read 0xad821c5 thru 0xad864d4 (0x4310 bytes) 2024-09-16T11:14:08.642 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:08.642 INFO:teuthology.orchestra.run.smithi162.stdout:3080 read 0x33604ed thru 0x3370411 (0xff25 bytes) 2024-09-16T11:14:08.665 INFO:teuthology.orchestra.run.smithi162.stdout:3081 trunc from 0xe6d5d3f to 0x13983b7 2024-09-16T11:14:08.682 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:08.676+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:08.703 INFO:teuthology.orchestra.run.smithi162.stdout:3082 write 0x5dcac85 thru 0x5dd213a (0x74b6 bytes) 2024-09-16T11:14:08.713 INFO:teuthology.orchestra.run.smithi162.stdout:3083 punch from 0x2773456 to 0x277635a, (0x2f04 bytes) 2024-09-16T11:14:08.714 INFO:teuthology.orchestra.run.smithi162.stdout:3084 write 0x55db564 thru 0x55dcc1c (0x16b9 bytes) 2024-09-16T11:14:08.717 INFO:teuthology.orchestra.run.smithi162.stdout:3085 read 0x1999688 thru 0x19a8e9f (0xf818 bytes) 2024-09-16T11:14:08.719 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:08.719 INFO:teuthology.orchestra.run.smithi162.stdout:3087 punch from 0x1baa23a to 0x1bb06ba, (0x6480 bytes) 2024-09-16T11:14:08.724 INFO:teuthology.orchestra.run.smithi162.stdout:3088 trunc from 0x5dd213b to 0x997029e 2024-09-16T11:14:08.733 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:08.733 INFO:teuthology.orchestra.run.smithi162.stdout:3092 read 0xa652e6 thru 0xa6768f (0x23aa bytes) 2024-09-16T11:14:08.733 INFO:teuthology.orchestra.run.smithi162.stdout:3093 read 0x4e1e2d9 thru 0x4e21b34 (0x385c bytes) 2024-09-16T11:14:08.734 INFO:teuthology.orchestra.run.smithi162.stdout:3095 punch from 0x1f15243 to 0x1f1a306, (0x50c3 bytes) 2024-09-16T11:14:08.734 INFO:teuthology.orchestra.run.smithi162.stdout:3096 read 0x5b14670 thru 0x5b16388 (0x1d19 bytes) 2024-09-16T11:14:08.740 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:08.740 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:08.741 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:08.741 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:08.751 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:08.752 INFO:teuthology.orchestra.run.smithi162.stdout:3097 writesame 0xe35cc8f thru 0xe360b78 data_size 0x1(0x3eea bytes) 2024-09-16T11:14:08.752 INFO:teuthology.orchestra.run.smithi162.stdout:3098 write 0x155acc4 thru 0x15609ad (0x5cea bytes) 2024-09-16T11:14:08.756 INFO:teuthology.orchestra.run.smithi162.stdout:3102 read 0xe83adb thru 0xe8ee2e (0xb354 bytes) 2024-09-16T11:14:08.779 INFO:teuthology.orchestra.run.smithi162.stdout:3105 trunc from 0xe360b79 to 0x2672bdd 2024-09-16T11:14:08.782 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:08.776+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:08.795 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:08.819 INFO:teuthology.orchestra.run.smithi162.stdout:3106 write 0x90e83a9 thru 0x90e9262 (0xeba bytes) 2024-09-16T11:14:08.825 INFO:teuthology.orchestra.run.smithi162.stdout:3107 write 0x1bbc3c0 thru 0x1bcaa14 (0xe655 bytes) 2024-09-16T11:14:08.835 INFO:teuthology.orchestra.run.smithi162.stdout:3108 read 0x65aecfb thru 0x65bb83a (0xcb40 bytes) 2024-09-16T11:14:08.839 INFO:teuthology.orchestra.run.smithi162.stdout:3109 trunc from 0x90e9263 to 0x65ce2e2 2024-09-16T11:14:08.861 INFO:teuthology.orchestra.run.smithi162.stdout:3111 trunc from 0x65ce2e2 to 0x276ea28 2024-09-16T11:14:08.871 INFO:teuthology.orchestra.run.smithi162.stdout:3112 read 0x81b28 thru 0x82ce6 (0x11bf bytes) 2024-09-16T11:14:08.872 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:08.872 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:08.872 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:08.872 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:08.872 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:08.884 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:08.881+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:08.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:08.886 INFO:teuthology.orchestra.run.smithi162.stdout:3114 writesame 0x5393a3b thru 0x539c7a0 data_size 0x9(0x8d66 bytes) 2024-09-16T11:14:08.886 INFO:teuthology.orchestra.run.smithi162.stdout:3116 writesame 0x1c4a2ac thru 0x1c4a50f data_size 0x9(0x264 bytes) 2024-09-16T11:14:08.899 INFO:teuthology.orchestra.run.smithi162.stdout:3117 trunc from 0x539c7a1 to 0x9613577 2024-09-16T11:14:08.909 INFO:teuthology.orchestra.run.smithi162.stdout:3118 writesame 0x79db3ff thru 0x79dcc3f data_size 0x1(0x1841 bytes) 2024-09-16T11:14:08.920 INFO:teuthology.orchestra.run.smithi162.stdout:3120 trunc from 0x9613577 to 0x7bfa1f1 2024-09-16T11:14:08.939 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:08.944 INFO:teuthology.orchestra.run.smithi162.stdout:3123 write 0xa9218c3 thru 0xa92e099 (0xc7d7 bytes) 2024-09-16T11:14:08.957 INFO:teuthology.orchestra.run.smithi162.stdout:3124 writesame 0x4efed9 thru 0x4fe43c data_size 0x6a(0xe564 bytes) 2024-09-16T11:14:08.961 INFO:teuthology.orchestra.run.smithi162.stdout:3127 punch from 0x8450561 to 0x84555b8, (0x5057 bytes) 2024-09-16T11:14:08.961 INFO:teuthology.orchestra.run.smithi162.stdout:3128 write 0x86b41df thru 0x86c3165 (0xef87 bytes) 2024-09-16T11:14:08.963 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:08.979 INFO:teuthology.orchestra.run.smithi162.stdout:3130 writesame 0xabe2aef thru 0xabe3aa6 data_size 0x4(0xfb8 bytes) 2024-09-16T11:14:08.979 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:08.979 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:08.984 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:08.981+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:08.985 INFO:teuthology.orchestra.run.smithi162.stdout:3140 trunc from 0xabe3aa7 to 0x6c59996 2024-09-16T11:14:08.996 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:09.002 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.002 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.008 INFO:teuthology.orchestra.run.smithi162.stdout:3141 punch from 0xb71861 to 0xb813ae, (0xfb4d bytes) 2024-09-16T11:14:09.008 INFO:teuthology.orchestra.run.smithi162.stdout:3143 read 0x5ba1c99 thru 0x5ba6e4c (0x51b4 bytes) 2024-09-16T11:14:09.008 INFO:teuthology.orchestra.run.smithi162.stdout:3144 write 0x7ad2b4 thru 0x7b8315 (0xb062 bytes) 2024-09-16T11:14:09.012 INFO:teuthology.orchestra.run.smithi162.stdout:3145 punch from 0x2111d82 to 0x21159e7, (0x3c65 bytes) 2024-09-16T11:14:09.012 INFO:teuthology.orchestra.run.smithi162.stdout:3147 write 0x5f54b5 thru 0x604842 (0xf38e bytes) 2024-09-16T11:14:09.013 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:09.016 INFO:teuthology.orchestra.run.smithi162.stdout:3148 write 0x3f7970c thru 0x3f858ef (0xc1e4 bytes) 2024-09-16T11:14:09.023 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:09.027 INFO:teuthology.orchestra.run.smithi162.stdout:3149 trunc from 0x6c59996 to 0x6fcf164 2024-09-16T11:14:09.035 INFO:teuthology.orchestra.run.smithi162.stdout:3150 write 0x68efae0 thru 0x68fbb40 (0xc061 bytes) 2024-09-16T11:14:09.045 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:09.048 INFO:teuthology.orchestra.run.smithi162.stdout:3151 trunc from 0x6fcf164 to 0x818ce65 2024-09-16T11:14:09.057 INFO:teuthology.orchestra.run.smithi162.stdout:3152 read 0x6f54675 thru 0x6f55852 (0x11de bytes) 2024-09-16T11:14:09.057 INFO:teuthology.orchestra.run.smithi162.stdout:3153 trunc from 0x818ce65 to 0x841537b 2024-09-16T11:14:09.073 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:09.074 INFO:teuthology.orchestra.run.smithi162.stdout:3154 trunc from 0x841537b to 0x28cd7af 2024-09-16T11:14:09.086 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:09.080+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:09.096 INFO:teuthology.orchestra.run.smithi162.stdout:3156 read 0xc855e thru 0xc921f (0xcc2 bytes) 2024-09-16T11:14:09.096 INFO:teuthology.orchestra.run.smithi162.stdout:3157 writesame 0xf30763 thru 0xf3f099 data_size 0x7(0xe937 bytes) 2024-09-16T11:14:09.098 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:09.102 INFO:teuthology.orchestra.run.smithi162.stdout:3158 write 0xdc687f thru 0xdcb502 (0x4c84 bytes) 2024-09-16T11:14:09.119 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.119 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.122 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:09.125 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:09.126 INFO:teuthology.orchestra.run.smithi162.stdout:3159 trunc from 0x28cd7af to 0xeaa060d 2024-09-16T11:14:09.133 INFO:teuthology.orchestra.run.smithi162.stdout:3161 writesame 0xd102497 thru 0xd10ec0b data_size 0x1(0xc775 bytes) 2024-09-16T11:14:09.136 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:09.163 INFO:teuthology.orchestra.run.smithi162.stdout:3162 writesame 0xbd6391 thru 0xbdca8a data_size 0x31(0x66fa bytes) 2024-09-16T11:14:09.177 INFO:teuthology.orchestra.run.smithi162.stdout:3164 writesame 0xed27821 thru 0xed2fbaf data_size 0x1(0x838f bytes) 2024-09-16T11:14:09.178 INFO:teuthology.orchestra.run.smithi162.stdout:3165 write 0x3d9ff64 thru 0x3da6826 (0x68c3 bytes) 2024-09-16T11:14:09.187 INFO:teuthology.orchestra.run.smithi162.stdout:3166 write 0x105d13d thru 0x1069517 (0xc3db bytes) 2024-09-16T11:14:09.188 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:09.180+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:09.190 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.198 INFO:teuthology.orchestra.run.smithi162.stdout:3168 trunc from 0xed2fbb0 to 0xb01d1ca 2024-09-16T11:14:09.216 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.216 INFO:teuthology.orchestra.run.smithi162.stdout:3170 writesame 0xa8a2da6 thru 0xa8a6b2a data_size 0x1(0x3d85 bytes) 2024-09-16T11:14:09.232 INFO:teuthology.orchestra.run.smithi162.stdout:3171 write 0xf89ee6 thru 0xf9659f (0xc6ba bytes) 2024-09-16T11:14:09.236 INFO:teuthology.orchestra.run.smithi162.stdout:3172 write 0x7e28a1a thru 0x7e29078 (0x65f bytes) 2024-09-16T11:14:09.242 INFO:teuthology.orchestra.run.smithi162.stdout:3173 read 0xab061e thru 0xab91ff (0x8be2 bytes) 2024-09-16T11:14:09.259 INFO:teuthology.orchestra.run.smithi162.stdout:3174 write 0xe2d8062 thru 0xe2df7dd (0x777c bytes) 2024-09-16T11:14:09.266 INFO:teuthology.orchestra.run.smithi162.stdout:3175 read 0x44014bb thru 0x4409030 (0x7b76 bytes) 2024-09-16T11:14:09.269 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.269 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.281 INFO:teuthology.orchestra.run.smithi162.stdout:3176 trunc from 0xe2df7de to 0x3e089fc 2024-09-16T11:14:09.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:09.288 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:09.285+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:09.303 INFO:teuthology.orchestra.run.smithi162.stdout:3177 write 0x36223f7 thru 0x3628bb3 (0x67bd bytes) 2024-09-16T11:14:09.311 INFO:teuthology.orchestra.run.smithi162.stdout:3178 read 0x3b0eef7 thru 0x3b1220a (0x3314 bytes) 2024-09-16T11:14:09.311 INFO:teuthology.orchestra.run.smithi162.stdout:3179 read 0x35eea1d thru 0x35f9c27 (0xb20b bytes) 2024-09-16T11:14:09.329 INFO:teuthology.orchestra.run.smithi162.stdout:3180 writesame 0x8077a46 thru 0x807d437 data_size 0x2(0x59f2 bytes) 2024-09-16T11:14:09.329 INFO:teuthology.orchestra.run.smithi162.stdout:3181 writesame 0x762b2ba thru 0x7636803 data_size 0xc3(0xb54a bytes) 2024-09-16T11:14:09.339 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.339 INFO:teuthology.orchestra.run.smithi162.stdout:3183 read 0x29f67c3 thru 0x29f80a8 (0x18e6 bytes) 2024-09-16T11:14:09.340 INFO:teuthology.orchestra.run.smithi162.stdout:3184 read 0x57e7012 thru 0x57f4694 (0xd683 bytes) 2024-09-16T11:14:09.340 INFO:teuthology.orchestra.run.smithi162.stdout:3185 read 0x4ec7d9f thru 0x4ed5b48 (0xddaa bytes) 2024-09-16T11:14:09.340 INFO:teuthology.orchestra.run.smithi162.stdout:3187 read 0x3d9778a thru 0x3d97e2e (0x6a5 bytes) 2024-09-16T11:14:09.340 INFO:teuthology.orchestra.run.smithi162.stdout:3188 read 0x6df78d4 thru 0x6e03b4e (0xc27b bytes) 2024-09-16T11:14:09.340 INFO:teuthology.orchestra.run.smithi162.stdout:3189 read 0x1debb0b thru 0x1df08cc (0x4dc2 bytes) 2024-09-16T11:14:09.341 INFO:teuthology.orchestra.run.smithi162.stdout:3190 write 0x63bf860 thru 0x63c38c8 (0x4069 bytes) 2024-09-16T11:14:09.349 INFO:teuthology.orchestra.run.smithi162.stdout:3192 punch from 0x39091ee to 0x3918b90, (0xf9a2 bytes) 2024-09-16T11:14:09.349 INFO:teuthology.orchestra.run.smithi162.stdout:3194 punch from 0x6194dde to 0x61a1a9f, (0xccc1 bytes) 2024-09-16T11:14:09.349 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.349 INFO:teuthology.orchestra.run.smithi162.stdout:3196 read 0xaf91df thru 0xb01022 (0x7e44 bytes) 2024-09-16T11:14:09.350 INFO:teuthology.orchestra.run.smithi162.stdout:3197 read 0x5904118 thru 0x590ee28 (0xad11 bytes) 2024-09-16T11:14:09.350 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.350 INFO:teuthology.orchestra.run.smithi162.stdout:3199 read 0x568a979 thru 0x569a648 (0xfcd0 bytes) 2024-09-16T11:14:09.350 INFO:teuthology.orchestra.run.smithi162.stdout:3200 write 0x3285c21 thru 0x3291919 (0xbcf9 bytes) 2024-09-16T11:14:09.357 INFO:teuthology.orchestra.run.smithi162.stdout:3201 read 0x5d6aeec thru 0x5d74136 (0x924b bytes) 2024-09-16T11:14:09.370 INFO:teuthology.orchestra.run.smithi162.stdout:3202 trunc from 0x807d438 to 0xf7ae3b 2024-09-16T11:14:09.389 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:09.385+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:09.397 INFO:teuthology.orchestra.run.smithi162.stdout:3203 write 0xa77c59 thru 0xa7f59a (0x7942 bytes) 2024-09-16T11:14:09.400 INFO:teuthology.orchestra.run.smithi162.stdout:3204 read 0x642df0 thru 0x64f87c (0xca8d bytes) 2024-09-16T11:14:09.401 INFO:teuthology.orchestra.run.smithi162.stdout:3205 punch from 0x87c1cc to 0x882a24, (0x6858 bytes) 2024-09-16T11:14:09.412 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.412 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.416 INFO:teuthology.orchestra.run.smithi162.stdout:3206 writesame 0x48f57ce thru 0x48f97b7 data_size 0x1b(0x3fea bytes) 2024-09-16T11:14:09.416 INFO:teuthology.orchestra.run.smithi162.stdout:3207 punch from 0x44aec8d to 0x44b4f1b, (0x628e bytes) 2024-09-16T11:14:09.422 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:09.426 INFO:teuthology.orchestra.run.smithi162.stdout:3208 writesame 0x55ba862 thru 0x55bb8da data_size 0x1(0x1079 bytes) 2024-09-16T11:14:09.443 INFO:teuthology.orchestra.run.smithi162.stdout:3213 write 0x9655ec7 thru 0x9660028 (0xa162 bytes) 2024-09-16T11:14:09.457 INFO:teuthology.orchestra.run.smithi162.stdout:3214 write 0x7b76da3 thru 0x7b7b4f7 (0x4755 bytes) 2024-09-16T11:14:09.463 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.463 INFO:teuthology.orchestra.run.smithi162.stdout:3216 read 0x9375ae6 thru 0x937cc14 (0x712f bytes) 2024-09-16T11:14:09.464 INFO:teuthology.orchestra.run.smithi162.stdout:3217 writesame 0x52da9ce thru 0x52e947c data_size 0x1(0xeaaf bytes) 2024-09-16T11:14:09.490 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:09.489+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:09.496 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.496 INFO:teuthology.orchestra.run.smithi162.stdout:3219 writesame 0x4835643 thru 0x484051d data_size 0x2b(0xaedb bytes) 2024-09-16T11:14:09.508 INFO:teuthology.orchestra.run.smithi162.stdout:3220 write 0x47b8f32 thru 0x47c548f (0xc55e bytes) 2024-09-16T11:14:09.518 INFO:teuthology.orchestra.run.smithi162.stdout:3221 write 0x1de30f3 thru 0x1de71d3 (0x40e1 bytes) 2024-09-16T11:14:09.524 INFO:teuthology.orchestra.run.smithi162.stdout:3222 write 0x2d55413 thru 0x2d5ca68 (0x7656 bytes) 2024-09-16T11:14:09.529 INFO:teuthology.orchestra.run.smithi162.stdout:3223 read 0x3fe806b thru 0x3ff5318 (0xd2ae bytes) 2024-09-16T11:14:09.541 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.541 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.543 INFO:teuthology.orchestra.run.smithi162.stdout:3226 write 0xab087dd thru 0xab0a77b (0x1f9f bytes) 2024-09-16T11:14:09.548 INFO:teuthology.orchestra.run.smithi162.stdout:3227 trunc from 0xab0a77c to 0xa9cd186 2024-09-16T11:14:09.551 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:09.561 INFO:teuthology.orchestra.run.smithi162.stdout:3229 read 0x1ef22af thru 0x1efb957 (0x96a9 bytes) 2024-09-16T11:14:09.562 INFO:teuthology.orchestra.run.smithi162.stdout:3230 read 0x6d27568 thru 0x6d2e3c2 (0x6e5b bytes) 2024-09-16T11:14:09.562 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.582 INFO:teuthology.orchestra.run.smithi162.stdout:3232 trunc from 0xa9cd186 to 0x900879 2024-09-16T11:14:09.591 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:09.584+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:09.608 INFO:teuthology.orchestra.run.smithi162.stdout:3234 read 0x5bdbd3 thru 0x5c06af (0x2add bytes) 2024-09-16T11:14:09.609 INFO:teuthology.orchestra.run.smithi162.stdout:3237 read 0x6bcd5c thru 0x6bd3b9 (0x65e bytes) 2024-09-16T11:14:09.609 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.629 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:14:09.634 INFO:teuthology.orchestra.run.smithi162.stdout:3239 write 0xaf5cb85 thru 0xaf6194f (0x4dcb bytes) 2024-09-16T11:14:09.644 INFO:teuthology.orchestra.run.smithi162.stdout:3241 read 0x175756d thru 0x1757fb7 (0xa4b bytes) 2024-09-16T11:14:09.651 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.651 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.658 INFO:teuthology.orchestra.run.smithi162.stdout:3242 write 0xd8ad789 thru 0xd8bc3f4 (0xec6c bytes) 2024-09-16T11:14:09.660 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:09.675 INFO:teuthology.orchestra.run.smithi162.stdout:3243 read 0xb90ea8e thru 0xb91c6cb (0xdc3e bytes) 2024-09-16T11:14:09.675 INFO:teuthology.orchestra.run.smithi162.stdout:3244 write 0x14dd3a2 thru 0x14df940 (0x259f bytes) 2024-09-16T11:14:09.677 INFO:teuthology.orchestra.run.smithi162.stdout:3245 writesame 0x1cdb34d thru 0x1cded97 data_size 0x1(0x3a4b bytes) 2024-09-16T11:14:09.692 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:09.689+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:09.700 INFO:teuthology.orchestra.run.smithi162.stdout:3246 write 0xe109c1e thru 0xe10ddab (0x418e bytes) 2024-09-16T11:14:09.703 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:09.703 INFO:teuthology.orchestra.run.smithi162.stdout:3252 read 0x64a46aa thru 0x64a544f (0xda6 bytes) 2024-09-16T11:14:09.703 INFO:teuthology.orchestra.run.smithi162.stdout:3253 punch from 0x71212d1 to 0x7121b4f, (0x87e bytes) 2024-09-16T11:14:09.703 INFO:teuthology.orchestra.run.smithi162.stdout:3254 writesame 0x11f3359 thru 0x1202d72 data_size 0x9(0xfa1a bytes) 2024-09-16T11:14:09.718 INFO:teuthology.orchestra.run.smithi162.stdout:3255 trunc from 0xe10ddac to 0x8b90305 2024-09-16T11:14:09.735 INFO:teuthology.orchestra.run.smithi162.stdout:3256 clone 24 order 18 su 2048 sc 13 2024-09-16T11:14:09.756 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.756 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.765 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:09.793 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:09.788+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:09.861 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.861 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:09.893 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:09.892+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:09.966 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:09.966 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:09.975 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:09.994 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:09.992+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:10.076 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.076 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.085 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:10.096 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:10.092+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:10.185 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.185 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.194 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:10.196 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:10.192+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:10.292 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.292 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.297 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:10.289+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:10.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:10.397 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:10.389+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:10.442 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.443 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.451 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:10.498 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:10.493+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:10.523 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:14:10.555 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.555 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.564 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:10.599 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:10.596+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:10.638 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone23 from 0x8b90305 (overlap 0x3b43e4) to 0x297c52 2024-09-16T11:14:10.699 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:10.693+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:10.711 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.712 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.721 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:10.756 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-16T11:14:10.800 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:10.793+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:10.815 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.815 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.824 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:10.901 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:10.897+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:10.934 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:10.934 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:10.944 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:11.002 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:11.000+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:11.036 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:11.036 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:11.046 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:11.103 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:11.100+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:11.140 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:11.148 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:11.148 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:11.158 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:11.162 INFO:teuthology.orchestra.run.smithi162.stdout:3257 write 0x3ffe370 thru 0x4002ed3 (0x4b64 bytes) 2024-09-16T11:14:11.187 INFO:teuthology.orchestra.run.smithi162.stdout:3258 clone 25 order 25 su 131072 sc 4 2024-09-16T11:14:11.204 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:11.197+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:11.264 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:11.264 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:11.274 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:11.291 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:11.306 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:11.301+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:11.308 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:11.325 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:11.341 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:11.362 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:11.387 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:11.389 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:11.389 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:11.389 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:11.389 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:11.389 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:11.398 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:11.406 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:11.401+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:11.408 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:11.436 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:11.438 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:11.439 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:11.507 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:11.504+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:11.509 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:11.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.345606022999998, 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:14:11.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.345570969000001, 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:14:11.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.518 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.345554535000002, 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:11.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:14:11.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.345538226999999, 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.523 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:14:11.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.345522661, 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.528 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.344788167000001, 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.532 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.344772139, 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.537 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.541 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.344762309, 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:14:11.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:14:11.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.341452112999999, 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.341428763, 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:11.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:14:11.554 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.341409463000002, 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.555 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:14:11.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.341391981000001, 2024-09-16T11:14:11.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:14:11.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:14:11.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.341361402, 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:11.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:14:11.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.341239585, 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:14:11.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:14:11.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.340798414999998, 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:14:11.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:11.577 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.340167543, 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:14:11.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.340139261000001, 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.582 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:14:11.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.340124032999999, 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:14:11.586 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:11.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:14:11.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.340111131, 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:14:11.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:14:11.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:14:11.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.0126457389999999, 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:11.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:11.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.603 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:11.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:11.609 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:11.604+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:11.664 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone24 from 0x8b90305 (overlap 0x8b90305) to 0x7d9b5ea 2024-09-16T11:14:11.709 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:11.708+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:11.726 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:11.726 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:11.726 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:11.726 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:11.736 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:11.810 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:11.805+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:11.876 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:11.876 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:11.876 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:11.876 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:11.876 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:11.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:11.910 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:11.908+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:11.932 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-16T11:14:11.959 INFO:teuthology.orchestra.run.smithi162.stdout:3259 write 0x5e105ce thru 0x5e153f1 (0x4e24 bytes) 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4276.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.161389049999997, 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001449746, 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:11.993 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830239+0000", 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830240+0000", 2024-09-16T11:14:11.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1800000000000001e-07 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830247+0000", 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5800000000000003e-06 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830259+0000", 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1418e-05 2024-09-16T11:14:11.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830271+0000", 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.198e-05 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830310+0000", 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9100999999999997e-05 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.830343+0000", 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3370000000000001e-05 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831651+0000", 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013075459999999999 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:12:54.831689+0000", 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8333e-05 2024-09-16T11:14:11.997 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.870814279999998, 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:11.998 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:11.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:14:12.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:14:12.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:10 4.1c e34/31 ondisk, result = 0)", 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.862405991999999, 2024-09-16T11:14:12.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014537900000000001, 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129222+0000", 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129224+0000", 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129232+0000", 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8630000000000008e-06 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129253+0000", 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0868000000000001e-05 2024-09-16T11:14:12.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129268+0000", 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5330999999999999e-05 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129342+0000", 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3946e-05 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129354+0000", 2024-09-16T11:14:12.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1864e-05 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129368+0000", 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3952e-05 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.008 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:12.009 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:12.011 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:12.005+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:12.112 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:12.108+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:12.164 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:12.164 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:12.165 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:12.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:12.214 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:12.212+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:12.264 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:12.264 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:12.264 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:12.264 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:12.264 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:12.272 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:12.317 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:12.316+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:12.417 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:12.413+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:12.419 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:12.419 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:12.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49775 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1286144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282602957000002, 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050091745, 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49775 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133156+0000", 2024-09-16T11:14:12.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1179999999999999e-06 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133159+0000", 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133178+0000", 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8692e-05 2024-09-16T11:14:12.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.182940+0000", 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049762038000000001 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183007+0000", 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7082000000000006e-05 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183247+0000", 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239581 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282578020999999, 2024-09-16T11:14:12.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:12.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282444278, 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:12.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282425889999999, 2024-09-16T11:14:12.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:12.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282125133000001, 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.442 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:12.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282095428000002, 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:12.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:12.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:12.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282082030000002, 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.453 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:12.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.282068226, 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.458 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:12.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.281693227000002, 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.463 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:12.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50035 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:12.468 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.268164919, 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049942324000000003, 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50035 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:12.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147595+0000", 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9769999999999999e-06 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147601+0000", 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147605+0000", 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197156+0000", 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049550679 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197231+0000", 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5853999999999996e-05 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197535+0000", 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000303861 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.268149196, 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:12.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.268127622000002, 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:12.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:12.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:12.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.484 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.268111123000001, 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:12.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:12.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.489 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.268095046999999, 2024-09-16T11:14:12.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:12.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:12.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.267711472999999, 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.497 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:12.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:12.501 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50291 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.255673399999999, 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049900104000000001, 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.502 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50291 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160084+0000", 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160085+0000", 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7000000000000004e-07 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160089+0000", 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6679999999999999e-06 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160100+0000", 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1311999999999999e-05 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160105+0000", 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.583e-06 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209651+0000", 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049546305999999998 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209730+0000", 2024-09-16T11:14:12.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9295999999999994e-05 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.209985+0000", 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000254169 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50292 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.507 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.255648574999999, 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050216150000000001, 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50292 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160110+0000", 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160117+0000", 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.362e-06 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160124+0000", 2024-09-16T11:14:12.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160126+0000", 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210014+0000", 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049887881000000002 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210108+0000", 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3356000000000005e-05 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210325+0000", 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021782400000000001 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.255629295999999, 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.513 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:12.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:12.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.255621094999999, 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:12.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.255613072999999, 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:12.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:12.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:12.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:12.528 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:12.530 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:12.531 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:12.521+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:12.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:14:12.630 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:12.630 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:12.631 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T11:14:12.631 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:14:12.631 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:14:12.631 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T11:14:12.632 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:12.625+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:12.640 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:12.640 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:12.650 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:12.731 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:12.725+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:12.748 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:12.748 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "" 2024-09-16T11:14:12.748 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:12.753 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:12.753 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:12.753 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:12.753 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:12.753 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:12.762 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:12.832 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:12.828+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15969 4.d 4:b69afd92:::rbd_data.11b58a761a59.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:12.912 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.828413079999997, 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044609673000000002, 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15969 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079092+0000", 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.17e-06 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079104+0000", 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079140+0000", 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5648000000000002e-05 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079152+0000", 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2636999999999999e-05 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079270+0000", 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011812200000000001 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079499+0000", 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022818 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.082238+0000", 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027396949999999999 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:12.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123504+0000", 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041265556000000002 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123526+0000", 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1971999999999999e-05 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123623+0000", 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7347000000000002e-05 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123700+0000", 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6786000000000006e-05 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15970 4.d 4:b7b5bf19:::rbd_data.11b58a761a59.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.828108417999999, 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044492102999999998, 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:12.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15970 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079397+0000", 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.793e-06 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079405+0000", 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079421+0000", 2024-09-16T11:14:12.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6201e-05 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079429+0000", 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.092788+0000", 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013358761 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.093013+0000", 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000225687 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.098959+0000", 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059459059999999999 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123735+0000", 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024775755999999999 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:12.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123753+0000", 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8065999999999999e-05 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123836+0000", 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-05 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123887+0000", 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0559000000000002e-05 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26656 4.9 4:95b731cd:::rbd_data.11b5499bda95.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.956603412, 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.054121295999999999, 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26656 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950902+0000", 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:12.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950914+0000", 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1724e-05 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950943+0000", 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9621e-05 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950956+0000", 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.238e-05 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951090+0000", 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013472399999999999 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951475+0000", 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000384378 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.964699+0000", 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013224217999999999 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004806+0000", 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.040106941 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004820+0000", 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3909e-05 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004895+0000", 2024-09-16T11:14:12.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4905000000000002e-05 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.005021+0000", 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012639300000000001 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81604 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1585152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84104862199999997, 2024-09-16T11:14:12.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041841556000000002, 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81604 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8400000000000004e-07 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066462+0000", 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0190000000000001e-06 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066469+0000", 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8410000000000002e-06 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066473+0000", 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9439999999999998e-06 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108177+0000", 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041703298 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108239+0000", 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2867000000000006e-05 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108296+0000", 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6903000000000001e-05 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81608 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1624064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:12.933 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84100577399999998, 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041955923999999999, 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81608 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066500+0000", 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0729999999999999e-06 2024-09-16T11:14:12.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066507+0000", 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6969999999999996e-06 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066512+0000", 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9309999999999999e-06 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108329+0000", 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041817528 2024-09-16T11:14:12.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108392+0000", 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2507000000000003e-05 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108453+0000", 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.937 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81611 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1619968~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84093283399999996, 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042054182000000002, 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81611 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066573+0000", 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4250000000000001e-06 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066580+0000", 2024-09-16T11:14:12.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066587+0000", 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.195e-06 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108481+0000", 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041894239 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108544+0000", 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3415000000000002e-05 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108625+0000", 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0451000000000006e-05 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81613 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1656832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84091172800000002, 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042194269999999999, 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81613 2024-09-16T11:14:12.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.61e-07 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066600+0000", 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8539999999999997e-06 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066607+0000", 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.221e-06 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066612+0000", 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.065e-06 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108659+0000", 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042047785999999997 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108724+0000", 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5049999999999996e-05 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108786+0000", 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1532999999999995e-05 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81614 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1667072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066617+0000", 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84088675099999999, 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042313200000000002, 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.947 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81614 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066617+0000", 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066617+0000", 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066618+0000", 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3769999999999999e-06 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066620+0000", 2024-09-16T11:14:12.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1619999999999998e-06 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066627+0000", 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8349999999999998e-06 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066634+0000", 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7240000000000003e-06 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108808+0000", 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042174365999999998 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108872+0000", 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3615000000000007e-05 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108930+0000", 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8121000000000001e-05 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81621 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1701888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84083738500000005, 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042420449999999998, 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81621 2024-09-16T11:14:12.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066667+0000", 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.111e-06 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066671+0000", 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0049999999999998e-06 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066674+0000", 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3e-06 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108963+0000", 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042289803000000001 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109026+0000", 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2595000000000006e-05 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109086+0000", 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0353000000000003e-05 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81623 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1738752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84082733899999995, 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042551654000000001, 2024-09-16T11:14:12.956 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81623 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066689+0000", 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2221999999999999e-05 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066693+0000", 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066695+0000", 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109108+0000", 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042413237999999999 2024-09-16T11:14:12.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109174+0000", 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5032999999999999e-05 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109228+0000", 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4151999999999999e-05 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81624 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1748992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84080571199999998, 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042688618999999997, 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81624 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.41e-07 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066699+0000", 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.511e-06 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066703+0000", 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066705+0000", 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.362e-06 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109258+0000", 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042552923999999999 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109323+0000", 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5094999999999999e-05 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109386+0000", 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2982000000000001e-05 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81632 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066707+0000", 2024-09-16T11:14:12.965 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84079588500000002, 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042840692999999999, 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81632 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066707+0000", 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066707+0000", 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066820+0000", 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112254 2024-09-16T11:14:12.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066822+0000", 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.148e-06 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066828+0000", 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3969999999999999e-06 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066831+0000", 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9569999999999998e-06 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109408+0000", 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042576935000000003 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109482+0000", 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4023000000000006e-05 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109548+0000", 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5979e-05 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81634 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1830912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066846+0000", 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.84065748200000001, 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042862465000000002, 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.970 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81634 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066846+0000", 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066846+0000", 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069619+0000", 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002772769 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069622+0000", 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:12.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069630+0000", 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069641+0000", 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750999999999999e-05 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109583+0000", 2024-09-16T11:14:12.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.039942377000000001 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109647+0000", 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3876999999999998e-05 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109708+0000", 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1181999999999998e-05 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81693 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1742848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83749277200000005, 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041817314000000001, 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81693 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.975 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:12.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070012+0000", 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.37e-06 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070016+0000", 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4640000000000002e-06 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070019+0000", 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3749999999999999e-06 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111697+0000", 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041677573000000002 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111765+0000", 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7972000000000006e-05 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111828+0000", 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3208999999999997e-05 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81703 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1824768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070061+0000", 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83744212799999995, 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041902087999999997, 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.979 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81703 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070061+0000", 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070061+0000", 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070062+0000", 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7700000000000005e-07 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070063+0000", 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3990000000000001e-06 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070067+0000", 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6109999999999998e-06 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070069+0000", 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.367e-06 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111850+0000", 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041780882999999998 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111909+0000", 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8736000000000001e-05 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111963+0000", 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4614999999999998e-05 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81708 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1581056~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83742835800000004, 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042045723, 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81708 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.46e-07 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070077+0000", 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5430000000000001e-06 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070082+0000", 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6330000000000004e-06 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070084+0000", 2024-09-16T11:14:12.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5189999999999999e-06 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111997+0000", 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041912798000000001 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112062+0000", 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5518000000000005e-05 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112121+0000", 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8366000000000003e-05 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81713 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1630208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83739962300000004, 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042163501999999999, 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81713 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070107+0000", 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.278e-06 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070113+0000", 2024-09-16T11:14:12.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070118+0000", 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3950000000000003e-06 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112144+0000", 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042025709000000001 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112209+0000", 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5811999999999999e-05 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112267+0000", 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7831999999999998e-05 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81718 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1662976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.992 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070185+0000", 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83731787499999999, 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042240643000000001, 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81718 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070185+0000", 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070185+0000", 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070186+0000", 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:14:12.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070191+0000", 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070197+0000", 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5199999999999997e-06 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070200+0000", 2024-09-16T11:14:12.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6290000000000002e-06 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112301+0000", 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042101765999999999 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112364+0000", 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2321000000000002e-05 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112426+0000", 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2389000000000002e-05 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81729 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1744896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83727747100000005, 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042348908999999997, 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:12.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81729 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:12.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070228+0000", 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3689999999999999e-06 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:12.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070233+0000", 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9860000000000002e-06 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070235+0000", 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4870000000000001e-06 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112448+0000", 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042213061000000003 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112517+0000", 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9138000000000001e-05 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112575+0000", 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7544999999999999e-05 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81908 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:13.001 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.83250860000000004, 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041871039999999998, 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81908 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.075001+0000", 2024-09-16T11:14:13.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4099999999999999e-06 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.075005+0000", 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2259999999999999e-06 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.075008+0000", 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6730000000000001e-06 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.116769+0000", 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041761570999999997 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.116818+0000", 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9079999999999998e-05 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.116866+0000", 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7583000000000002e-05 2024-09-16T11:14:13.005 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.006 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.006 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.006 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:13.007 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:13.008 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:13.001+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:13.108 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:13.101+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:13.111 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:13.111 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:13.111 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:13.111 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:13.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:13.210 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:13.208+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:13.266 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:13.266 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:13.266 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:13.266 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:13.266 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:13.275 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:13.311 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:13.305+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:13.378 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:13.378 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:13.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:13.378 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.511332967999998, 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:13.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:13.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343361319, 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:13.383 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:13.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:13.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343349757999999, 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:13.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:13.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:13.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343336556000001, 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:13.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:13.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343324116000002, 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:13.396 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:13.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:13.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343310604999999, 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:13.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343299689999998, 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.405 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:13.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.409 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343273772, 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:13.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.413 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343258318, 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:13.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:13.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343248431999999, 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:13.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:13.422 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343238669000002, 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:13.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343228492000001, 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.427 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:13.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343212009999998, 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:13.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:13.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343197313000001, 2024-09-16T11:14:13.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:13.439 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343178869999999, 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:13.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:13.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343161921, 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:13.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:13.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:13.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343142707999998, 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:13.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:13.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343125491999999, 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:13.452 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343104237999999, 2024-09-16T11:14:13.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.460 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.343084269999999, 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:13.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:13.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:13.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:13.464 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:13.465 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:13.467 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:13.467 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:13.457+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:13.553 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:14:13.566 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:13.561+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:13.612 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:13.618 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:13.618 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:13.618 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:13.618 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:13.628 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:13.631 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:13.650 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:13.668 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:13.661+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:13.676 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:13.696 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:13.716 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:13.732 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:13.732 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:13.737 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:13.741 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:13.758 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:13.760 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:13.769 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:13.768+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:13.870 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:13.861+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:13.885 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:13.885 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:13.894 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:13.971 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:13.968+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:13.991 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:13.992 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.001 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:14.072 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:14.065+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:14.106 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.106 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.115 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:14.173 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:14.172+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:14.218 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.218 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.227 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:14.251 INFO:teuthology.orchestra.run.smithi162.stdout:3260 punch from 0x1b6ef1 to 0x1c063b, (0x974a bytes) 2024-09-16T11:14:14.251 INFO:teuthology.orchestra.run.smithi162.stdout:3261 read 0x145f1b9 thru 0x1466748 (0x7590 bytes) 2024-09-16T11:14:14.268 INFO:teuthology.orchestra.run.smithi162.stdout:3263 punch from 0x2d3932b to 0x2d3a277, (0xf4c bytes) 2024-09-16T11:14:14.274 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:14.265+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:14.376 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:14.372+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:14.386 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.386 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.400 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:14.476 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:14.469+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:14.559 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.559 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.571 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:14.576 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:14.572+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:14.630 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:14.630 INFO:teuthology.orchestra.run.smithi162.stdout:3267 read 0x37a18bd thru 0x37a4455 (0x2b99 bytes) 2024-09-16T11:14:14.640 INFO:teuthology.orchestra.run.smithi162.stdout:3268 clone 26 order 21 su 131072 sc 9 2024-09-16T11:14:14.678 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:14.676+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:14.680 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.680 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:14.779 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:14.773+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:14.813 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.813 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:14.880 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:14.877+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:14.907 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:14.907 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:14.916 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:14.980 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:14.976+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:15.009 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.009 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.018 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:15.081 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:15.073+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:15.111 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.111 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.120 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:15.182 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:15.180+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:15.214 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.214 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.222 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:15.282 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:15.281+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:15.364 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.365 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:15.383 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:15.380+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:15.469 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.469 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.477 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:15.521 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:15.520+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:15.566 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:14:15.585 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:15.581+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:15.619 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.619 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.628 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:15.685 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:15.684+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:15.718 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.718 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.726 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:15.769 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:15.786 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:15.781+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:15.839 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.839 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:15.886 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:15.884+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:15.909 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:15.927 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:15.948 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:15.949 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:14:15.955 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:15.955 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:15.965 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:15.969 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:15.987 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:16.006 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:16.010 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:16.005+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:16.043 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:16.061 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:16.061 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:16.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:16.061 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:16.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.062 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15969 4.d 4:b69afd92:::rbd_data.11b58a761a59.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:14:16.062 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:16.062 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 40.978303922000002, 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044609673000000002, 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15969 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079090+0000", 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079092+0000", 2024-09-16T11:14:16.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.17e-06 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079104+0000", 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079140+0000", 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5648000000000002e-05 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079152+0000", 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2636999999999999e-05 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079270+0000", 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011812200000000001 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079499+0000", 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022818 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.082238+0000", 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027396949999999999 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123504+0000", 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041265556000000002 2024-09-16T11:14:16.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123526+0000", 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1971999999999999e-05 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123623+0000", 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7347000000000002e-05 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123700+0000", 2024-09-16T11:14:16.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6786000000000006e-05 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:15970 4.d 4:b7b5bf19:::rbd_data.11b58a761a59.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 40.977999259999997, 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.044492102999999998, 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 15970 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079395+0000", 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079397+0000", 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.793e-06 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079405+0000", 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9899999999999997e-06 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079421+0000", 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6201e-05 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.079429+0000", 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.092788+0000", 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013358761 2024-09-16T11:14:16.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.093013+0000", 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000225687 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.098959+0000", 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0059459059999999999 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123735+0000", 2024-09-16T11:14:16.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024775755999999999 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123753+0000", 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8065999999999999e-05 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123836+0000", 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3350000000000007e-05 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:35.123887+0000", 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0559000000000002e-05 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:26656 4.9 4:95b731cd:::rbd_data.11b5499bda95.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.106494253999998, 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.054121295999999999, 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 26656 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950900+0000", 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950902+0000", 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950914+0000", 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1724e-05 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950943+0000", 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9621e-05 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.950956+0000", 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.238e-05 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951090+0000", 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013472399999999999 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.951475+0000", 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000384378 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:38.964699+0000", 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013224217999999999 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004806+0000", 2024-09-16T11:14:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.040106941 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004820+0000", 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3909e-05 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.004895+0000", 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4905000000000002e-05 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:39.005021+0000", 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012639300000000001 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81604 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1585152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9909394640000002, 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041841556000000002, 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.081 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81604 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066455+0000", 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8400000000000004e-07 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066462+0000", 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0190000000000001e-06 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066469+0000", 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8410000000000002e-06 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066473+0000", 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9439999999999998e-06 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108177+0000", 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041703298 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108239+0000", 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2867000000000006e-05 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108296+0000", 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6903000000000001e-05 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.085 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81608 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1624064~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9908966160000001, 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041955923999999999, 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81608 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066498+0000", 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066500+0000", 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0729999999999999e-06 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066507+0000", 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6969999999999996e-06 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066512+0000", 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9309999999999999e-06 2024-09-16T11:14:16.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108329+0000", 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041817528 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108392+0000", 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2507000000000003e-05 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108453+0000", 2024-09-16T11:14:16.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1592000000000003e-05 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81611 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1619968~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9908236759999998, 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042054182000000002, 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81611 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066571+0000", 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7599999999999997e-07 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066573+0000", 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4250000000000001e-06 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066580+0000", 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9809999999999997e-06 2024-09-16T11:14:16.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066587+0000", 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.195e-06 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108481+0000", 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041894239 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108544+0000", 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3415000000000002e-05 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108625+0000", 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0451000000000006e-05 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81613 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1656832~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:16.094 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.99080257, 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042194269999999999, 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81613 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066592+0000", 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.61e-07 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066600+0000", 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8539999999999997e-06 2024-09-16T11:14:16.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066607+0000", 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.221e-06 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066612+0000", 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.065e-06 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108659+0000", 2024-09-16T11:14:16.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042047785999999997 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108724+0000", 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5049999999999996e-05 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108786+0000", 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1532999999999995e-05 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81614 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1667072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066617+0000", 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9907775929999998, 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042313200000000002, 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81614 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066617+0000", 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066617+0000", 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066618+0000", 2024-09-16T11:14:16.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3769999999999999e-06 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066620+0000", 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1619999999999998e-06 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066627+0000", 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8349999999999998e-06 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066634+0000", 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7240000000000003e-06 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108808+0000", 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042174365999999998 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108872+0000", 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3615000000000007e-05 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108930+0000", 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8121000000000001e-05 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81621 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1701888~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.990728227, 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042420449999999998, 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81621 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066666+0000", 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066667+0000", 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.111e-06 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066671+0000", 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0049999999999998e-06 2024-09-16T11:14:16.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066674+0000", 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3e-06 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.108963+0000", 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042289803000000001 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109026+0000", 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2595000000000006e-05 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109086+0000", 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0353000000000003e-05 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81623 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1738752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9907181810000001, 2024-09-16T11:14:16.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042551654000000001, 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81623 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066676+0000", 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066689+0000", 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2221999999999999e-05 2024-09-16T11:14:16.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066693+0000", 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2289999999999997e-06 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066695+0000", 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109108+0000", 2024-09-16T11:14:16.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042413237999999999 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109174+0000", 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5032999999999999e-05 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109228+0000", 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4151999999999999e-05 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81624 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1748992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9906965539999999, 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042688618999999997, 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81624 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066698+0000", 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.41e-07 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066699+0000", 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.511e-06 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066703+0000", 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066705+0000", 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.362e-06 2024-09-16T11:14:16.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109258+0000", 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042552923999999999 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109323+0000", 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5094999999999999e-05 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109386+0000", 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2982000000000001e-05 2024-09-16T11:14:16.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81632 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066707+0000", 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9906867269999999, 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042840692999999999, 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.116 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81632 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066707+0000", 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066707+0000", 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066820+0000", 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112254 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066822+0000", 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.148e-06 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066828+0000", 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3969999999999999e-06 2024-09-16T11:14:16.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066831+0000", 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9569999999999998e-06 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109408+0000", 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042576935000000003 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109482+0000", 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4023000000000006e-05 2024-09-16T11:14:16.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109548+0000", 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5979e-05 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81634 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1830912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.066846+0000", 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9905483240000001, 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042862465000000002, 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.120 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81634 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066846+0000", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.066846+0000", 2024-09-16T11:14:16.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069619+0000", 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002772769 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069622+0000", 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069630+0000", 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:16.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.069641+0000", 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750999999999999e-05 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109583+0000", 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.039942377000000001 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109647+0000", 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3876999999999998e-05 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.109708+0000", 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1181999999999998e-05 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81693 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1742848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9873836140000001, 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041817314000000001, 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81693 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070011+0000", 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5100000000000001e-07 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070012+0000", 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.37e-06 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070016+0000", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4640000000000002e-06 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070019+0000", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3749999999999999e-06 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111697+0000", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041677573000000002 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111765+0000", 2024-09-16T11:14:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7972000000000006e-05 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111828+0000", 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3208999999999997e-05 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81703 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1824768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070061+0000", 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9873329699999998, 2024-09-16T11:14:16.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041902087999999997, 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81703 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070061+0000", 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070061+0000", 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070062+0000", 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7700000000000005e-07 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070063+0000", 2024-09-16T11:14:16.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3990000000000001e-06 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070067+0000", 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6109999999999998e-06 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070069+0000", 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.367e-06 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111850+0000", 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041780882999999998 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111909+0000", 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8736000000000001e-05 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111963+0000", 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4614999999999998e-05 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.132 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81708 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1581056~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9873192, 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042045723, 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.133 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81708 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070075+0000", 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.46e-07 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070077+0000", 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5430000000000001e-06 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070082+0000", 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6330000000000004e-06 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070084+0000", 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5189999999999999e-06 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.111997+0000", 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041912798000000001 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112062+0000", 2024-09-16T11:14:16.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5518000000000005e-05 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112121+0000", 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8366000000000003e-05 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81713 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1630208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9872904650000001, 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042163501999999999, 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81713 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070104+0000", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070107+0000", 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.278e-06 2024-09-16T11:14:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070113+0000", 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070118+0000", 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3950000000000003e-06 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112144+0000", 2024-09-16T11:14:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042025709000000001 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112209+0000", 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5811999999999999e-05 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112267+0000", 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7831999999999998e-05 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81718 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1662976~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070185+0000", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9872087170000001, 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042240643000000001, 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81718 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070185+0000", 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070185+0000", 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070186+0000", 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070191+0000", 2024-09-16T11:14:16.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6999999999999996e-06 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070197+0000", 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5199999999999997e-06 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070200+0000", 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6290000000000002e-06 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112301+0000", 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042101765999999999 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112364+0000", 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2321000000000002e-05 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112426+0000", 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2389000000000002e-05 2024-09-16T11:14:16.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81729 4.d 4:b7868ff8:::rbd_data.11b578dab209.0000000000000000:1b [read 1744896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9871683129999997, 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042348908999999997, 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81729 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:16.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070226+0000", 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2300000000000002e-07 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070228+0000", 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3689999999999999e-06 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070233+0000", 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9860000000000002e-06 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.070235+0000", 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4870000000000001e-06 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112448+0000", 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.042213061000000003 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112517+0000", 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9138000000000001e-05 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.112575+0000", 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7544999999999999e-05 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:81908 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1820672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-16T11:14:16.150 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 3.9823994420000002, 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041871039999999998, 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 81908 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:16.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.074995+0000", 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.075001+0000", 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4099999999999999e-06 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.075005+0000", 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2259999999999999e-06 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.075008+0000", 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6730000000000001e-06 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.116769+0000", 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.041761570999999997 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.116818+0000", 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9079999999999998e-05 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:12.116866+0000", 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7583000000000002e-05 2024-09-16T11:14:16.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.155 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.155 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.155 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.155 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.155 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:16.156 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:16.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:16.158 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:16.152+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:16.257 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:16.249+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:16.259 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:16.260 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:16.260 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:16.260 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:16.269 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:16.358 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:16.356+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:16.414 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:16.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:16.415 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:16.415 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:16.415 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:16.424 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:16.459 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:16.453+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:16.527 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 55.660842270000003, 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:16.528 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:16.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492870621000002, 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.533 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:16.537 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492859060000001, 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.538 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:16.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:16.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492845857999999, 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.543 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:16.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492833418, 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:16.547 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:16.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:16.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492819907000001, 2024-09-16T11:14:16.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:16.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492808992000001, 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:16.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492783073999998, 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:16.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492767619999999, 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:16.567 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:16.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492757734000001, 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.576 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:16.577 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492747971, 2024-09-16T11:14:16.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:16.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.579 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.583 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492737794, 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492721312, 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:16.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:16.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492706614999999, 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.592 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492688172000001, 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:16.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:16.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492671222999999, 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.601 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:16.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.49265201, 2024-09-16T11:14:16.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.609 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492634794000001, 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:16.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492613540000001, 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:16.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.492593572000001, 2024-09-16T11:14:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:16.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:16.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:16.623 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:16.624 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:16.624 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:16.617+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:16.725 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:16.717+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:16.772 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:16.772 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:16.773 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:16.773 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:16.784 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:16.830 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:16.825+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:16.882 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:16.882 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:16.893 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:16.926 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:16.921+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.811387159999999, 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.001 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:17.005 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.807028633000002, 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.006 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:17.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:17.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50301 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:17.010 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.803409751, 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11841462, 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50301 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193761+0000", 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193762+0000", 2024-09-16T11:14:17.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7399999999999998e-07 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193764+0000", 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193771+0000", 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.494e-06 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193775+0000", 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3619999999999999e-06 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311756+0000", 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11798077999999999 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.311868+0000", 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112605 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312176+0000", 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030757000000000001 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.803390103000002, 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.019 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.803371317, 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:17.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:17.024 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.803353993999998, 2024-09-16T11:14:17.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:17.025 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.025 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.025 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.025 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:17.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.803336524999999, 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:17.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799496066, 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:17.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799485951000001, 2024-09-16T11:14:17.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.043 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799476070000001, 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:17.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:17.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:17.048 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799466893999998, 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:17.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:17.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:17.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799450827000001, 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.058 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799434431999998, 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.059 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:17.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:17.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.064 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799424857999998, 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:17.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.068 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.799415322000002, 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.069 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:17.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50319 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.795368927999998, 2024-09-16T11:14:17.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118443188, 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50319 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201804+0000", 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-06 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201807+0000", 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201809+0000", 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320222+0000", 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118412609 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320245+0000", 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3805999999999999e-05 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.795358664999998, 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:17.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:17.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:17.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:17.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:17.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.795349283, 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.083 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:17.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.795340625000001, 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:17.087 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.795331391000001, 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:17.091 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:17.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.096 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.096 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:17.097 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:17.098 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:17.093+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:17.198 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:17.196+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:17.246 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:17.246 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:17.246 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:17.256 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:17.298 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:17.296+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:17.355 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:17.355 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:17.356 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:17.356 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:17.356 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:17.364 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:17.399 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:17.396+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:17.469 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:17.469 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:17.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:17.469 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.688125311999997, 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.300199612, 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.477 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:17.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:17.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:17.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:17.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:17.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:17.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.300048282000006, 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.485 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:17.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:17.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299996796000002, 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.492 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:17.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:17.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:17.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:17.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299926647999996, 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.499 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:17.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:17.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.505 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299754178000001, 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:17.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:17.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:17.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:17.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:17.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:17.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:17.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:17.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:17.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:17.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:17.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.514 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.516 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.516 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:17.516 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299695856, 2024-09-16T11:14:17.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:17.516 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.516 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.517 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.517 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:17.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.517 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:17.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:17.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:17.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:17.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:17.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:17.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:17.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:17.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:17.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:17.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:17.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:17.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:17.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:17.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:17.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:17.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:17.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:17.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:17.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:17.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:17.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:17.529 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.529 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.529 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.530 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.530 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:17.530 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299543736000004, 2024-09-16T11:14:17.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:17.530 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:17.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:17.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299365217000002, 2024-09-16T11:14:17.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:17.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:17.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:17.542 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.299332243999999, 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:17.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:17.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.747144519000003, 2024-09-16T11:14:17.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:17.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:17.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.747015722, 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:17.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:17.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.746957377000001, 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.560 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:17.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:17.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.746902988000002, 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:17.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:17.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.746768681000006, 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.572 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:17.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.965613312000002, 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:17.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:17.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:17.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.965525999999997, 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:17.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:17.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.589 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.965287872000005, 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:17.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:17.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:17.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.284907863000001, 2024-09-16T11:14:17.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:17.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:14:17.601 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.576715427, 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:17.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:17.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.605 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:17.606 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:17.606 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:17.607 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:17.605+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:17.607 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:17.706 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:17.706 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:17.706 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:17.708 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:17.704+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:17.714 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:17.756 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:17.757 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:17.757 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-16T11:14:17.757 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-16T11:14:17.808 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:17.808 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:17.808 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:17.809 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:17.808+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:17.816 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:17.909 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.909 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.730575143999999, 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:17.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:17.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.914 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.730497401000001, 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:17.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.728131386000001, 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:17.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:17.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.924 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.728111869999999, 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:17.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.726974366, 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:17.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:17.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.726958467000003, 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:17.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:17.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:17.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.726934213, 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:17.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.726918953999999, 2024-09-16T11:14:17.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:17.950 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.726903634999999, 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:17.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:17.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724881230999998, 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724866732999999, 2024-09-16T11:14:17.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:17.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:17.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724852400000003, 2024-09-16T11:14:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.970 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.971 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:17.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:17.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:17.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724838343000002, 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.983 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724824486999999, 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724810611999999, 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:17.988 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:17.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724796845, 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.993 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:17.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:17.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:17.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724749336000002, 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:17.998 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:17.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:18.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.724463518, 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:18.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:18.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:18.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:18.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:18.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.008 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.631799702000002, 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.009 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:18.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:18.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:18.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:18.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:18.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:18.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:18.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:18.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:18.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:18.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:18.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.016 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.017 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.017 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:18.017 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:18.018 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.631650520000001, 2024-09-16T11:14:18.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:18.018 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.018 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:18.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.018 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.019 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.019 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:18.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.019 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:18.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:18.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:18.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:18.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:18.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:18.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:18.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:18.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:18.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:18.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:18.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:18.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:18.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:18.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:18.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:18.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:18.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:18.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:18.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:18.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:18.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:18.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:18.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:18.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:18.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:18.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:18.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:18.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.032 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.032 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.032 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.032 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:18.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:14:18.035 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:18.029+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:18.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:18.040 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:14:18.036+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e76 definitely_dead 0 2024-09-16T11:14:18.126 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:18.126 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:18.126 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:18.126 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:18.134 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:18.135 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:18.132+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:18.159 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:18.224 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:18.225 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:18.233 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:18.236 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:18.229+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:18.292 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.158019604, 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:14:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157984549999998, 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.330 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157968115999999, 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:14:18.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157951808, 2024-09-16T11:14:18.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:18.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157936242000002, 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:14:18.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:14:18.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157201747999999, 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:18.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:18.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:14:18.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:14:18.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:14:18.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:14:18.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:14:18.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157185720000001, 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:14:18.353 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:14:18.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:14:18.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:14:18.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.157175890000001, 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.360 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:14:18.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153865694, 2024-09-16T11:14:18.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:14:18.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153842344000001, 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.369 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:14:18.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:14:18.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.373 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153823043999999, 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:18.378 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153805561999999, 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:14:18.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:14:18.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:18.383 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153774983000002, 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:14:18.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:14:18.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:18.388 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153653166000002, 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.153211996, 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.152581124000001, 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:14:18.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.402 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.152552841999999, 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:14:18.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.152537614, 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:14:18.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:14:18.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.152524712000002, 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:14:18.414 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:18.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:18.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:18.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:14:18.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:14:18.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:14:18.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:14:18.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:14:18.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:14:18.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:18.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:14:18.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:14:18.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:14:18.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:14:18.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.423 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.423 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:18.424 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:18.424 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.825059319999999, 2024-09-16T11:14:18.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:18.424 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:18.424 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:18.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:18.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:18.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:18.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:18.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:18.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:18.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:18.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:18.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:18.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:18.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:18.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:18.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:18.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:18.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:18.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:18.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:18.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:18.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:18.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:18.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:18.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:18.438 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:18.440 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:18.441 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:18.441 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:18.441 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:18.441 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:18.441 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:18.442 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:18.433+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:18.443 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:18.470 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:18.472 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:18.473 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:18.540 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:18.536+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:18.543 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:18.543 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:18.551 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:18.640 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:18.640 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:18.642 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:18.640+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:18.648 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:18.743 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:18.740+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:18.788 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:18.788 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:18.797 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:18.844 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:18.841+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:18.891 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:18.891 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:18.899 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:18.945 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:18.937+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:18.997 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:18.998 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.007 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:19.046 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:19.044+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:19.059 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:14:19.061 INFO:teuthology.orchestra.run.smithi162.stderr:marked in osd.2. 2024-09-16T11:14:19.074 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-16T11:14:19.112 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.112 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:19.147 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:19.144+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:19.229 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.229 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.240 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:19.247 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:19.244+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:19.338 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.338 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.347 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:19.348 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:19.344+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:19.444 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.445 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.449 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:19.441+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:19.454 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:19.550 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:19.548+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:19.552 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.552 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.562 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:19.651 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:19.645+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:19.663 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.663 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.672 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:19.752 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:19.748+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:19.815 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.815 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.824 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:19.853 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:19.852+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:19.922 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:19.922 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:19.932 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:19.954 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:19.945+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:20.031 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:20.031 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:20.040 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:20.086 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:20.081+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:20.155 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:20.152+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:20.186 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:20.186 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:20.195 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:20.256 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:20.252+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:20.297 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:20.297 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:20.305 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:20.357 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:20.356+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:20.406 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:20.406 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:20.415 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:20.458 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:20.456+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:20.488 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:20.513 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:20.513 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:20.522 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:20.560 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:20.553+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:20.644 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:20.647 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:20.647 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:20.647 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:20.657 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:20.659 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:20.653+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:20.663 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:20.690 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:20.710 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:20.732 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:20.743 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:20.744 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:20.752 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:20.755 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:20.760 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:20.753+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:20.778 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:20.799 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:20.802 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:20.802 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:20.802 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:20.803 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:20.803 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:20.803 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:20.803 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:20.803 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.852 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.673586362999998, 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.67350862, 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:20.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:20.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.671142605, 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.862 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:20.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:20.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.867 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.671123088999998, 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:20.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:20.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.669985584999999, 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.669969686000002, 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.877 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:20.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.669945431999999, 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.881 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:20.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.669930172999997, 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:20.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.669914853999998, 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.890 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667892449999997, 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:20.897 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:20.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:20.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:20.902 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667877951999998, 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.903 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:20.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:20.907 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667863619000002, 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:20.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:20.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:20.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667849562000001, 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:20.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667835705999998, 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:20.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667821830999998, 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.921 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:20.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667808063999999, 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:20.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:20.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667760555000001, 2024-09-16T11:14:20.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:20.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:20.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.667474736999999, 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:20.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.574810921000001, 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:20.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:20.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:20.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:20.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 36.574661739, 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:20.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:20.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:20.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:20.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:20.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:20.952 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:20.953 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:20.954 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:20.952+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:21.054 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:21.049+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:21.070 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:14:21.120 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:21.121 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:21.121 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:21.133 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:21.155 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:21.149+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:21.232 INFO:teuthology.orchestra.run.smithi162.stdout:flattening image image_client.0-clone25 2024-09-16T11:14:21.237 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:21.238 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:21.248 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:21.256 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:21.252+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50256 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.194823696, 2024-09-16T11:14:21.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064094252000000004, 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50256 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159713+0000", 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2300000000000001e-07 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159732+0000", 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8604e-05 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159739+0000", 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9140000000000002e-06 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159743+0000", 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-06 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223521+0000", 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063778082 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223567+0000", 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5930999999999998e-05 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223807+0000", 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000240196 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50257 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.194788641999999, 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064578426999999994, 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50257 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159748+0000", 2024-09-16T11:14:21.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.51e-07 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159750+0000", 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.159e-06 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159756+0000", 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.993e-06 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159760+0000", 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6990000000000001e-06 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223842+0000", 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064081549000000002 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.223950+0000", 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108393 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224326+0000", 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037608299999999998 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50258 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.194772208000003, 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064981707999999999, 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50258 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159764+0000", 2024-09-16T11:14:21.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159765+0000", 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159767+0000", 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1440000000000001e-06 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159772+0000", 2024-09-16T11:14:21.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6149999999999996e-06 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159777+0000", 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.42e-06 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224374+0000", 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064597527000000002 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224443+0000", 2024-09-16T11:14:21.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8923000000000004e-05 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224746+0000", 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030273499999999999 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50259 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:21.371 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.194755899999997, 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067691050000000003, 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50259 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:21.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159781+0000", 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159783+0000", 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3470000000000001e-06 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159789+0000", 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5650000000000001e-06 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159792+0000", 2024-09-16T11:14:21.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6619999999999998e-06 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224791+0000", 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064998711000000001 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.224925+0000", 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013381499999999999 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227472+0000", 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025465589999999999 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50260 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.194740334000002, 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068059403000000004, 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50260 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.159796+0000", 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160494+0000", 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069822199999999997 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160501+0000", 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160512+0000", 2024-09-16T11:14:21.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1511999999999999e-05 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160517+0000", 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5399999999999997e-06 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227503+0000", 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066986367000000005 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227562+0000", 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9466000000000003e-05 2024-09-16T11:14:21.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227855+0000", 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00029305500000000002 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50261 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1433600~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.194005840000003, 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068378928000000005, 2024-09-16T11:14:21.380 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50261 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160531+0000", 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0399999999999996e-07 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160534+0000", 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.542e-06 2024-09-16T11:14:21.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160541+0000", 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1330000000000001e-06 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160544+0000", 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7269999999999999e-06 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.227908+0000", 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067364687000000006 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228015+0000", 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010656 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228910+0000", 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00089467500000000005 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50262 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.193989811999998, 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068761693999999998, 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50262 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160547+0000", 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160548+0000", 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4309999999999999e-06 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160552+0000", 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2260000000000001e-06 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160554+0000", 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6879999999999999e-06 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.228942+0000", 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.068387707000000006 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229034+0000", 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1848000000000006e-05 2024-09-16T11:14:21.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229308+0000", 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027448499999999999 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50263 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:21.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.193979982000002, 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.069210840999999995, 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50263 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:21.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160556+0000", 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163834+0000", 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003277393 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163840+0000", 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2659999999999998e-06 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163853+0000", 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2896e-05 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163859+0000", 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2820000000000002e-06 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229344+0000", 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065484834000000006 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229461+0000", 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116894 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229767+0000", 2024-09-16T11:14:21.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030627600000000002 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50264 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190669786000001, 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066270461000000003, 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50264 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163867+0000", 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999998e-07 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163870+0000", 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.835e-06 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163879+0000", 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9220000000000003e-06 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163883+0000", 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3789999999999999e-06 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229795+0000", 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.065911174000000003 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.229871+0000", 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6206999999999997e-05 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230137+0000", 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026636500000000002 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50265 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190646436000002, 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066722297, 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50265 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163890+0000", 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9500000000000003e-07 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163893+0000", 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163901+0000", 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6399999999999997e-06 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163904+0000", 2024-09-16T11:14:21.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230185+0000", 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066280690000000003 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230303+0000", 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011784499999999999 2024-09-16T11:14:21.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230612+0000", 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030968499999999997 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50266 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:21.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190627136000003, 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188293999999996, 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50266 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:21.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163909+0000", 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163910+0000", 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163912+0000", 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9860000000000001e-06 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163919+0000", 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4150000000000002e-06 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163923+0000", 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7560000000000001e-06 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230659+0000", 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.066736580000000004 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.230774+0000", 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000114973 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231098+0000", 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032316000000000002 2024-09-16T11:14:21.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50267 4.12 4:48d04998:::rbd_data.11b5eaebe1b2.0000000000000000:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190609653999999, 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067528956000000001, 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.408 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50267 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:21.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163927+0000", 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.03e-07 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163930+0000", 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.334e-06 2024-09-16T11:14:21.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163936+0000", 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0209999999999996e-06 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163939+0000", 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3450000000000002e-06 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231127+0000", 2024-09-16T11:14:21.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067188039000000005 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231193+0000", 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6322999999999995e-05 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231456+0000", 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000262491 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50368 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190579075000002, 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067758279000000005, 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50368 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.413 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163957+0000", 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163958+0000", 2024-09-16T11:14:21.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4199999999999996e-07 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163961+0000", 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9220000000000001e-06 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163967+0000", 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.163971+0000", 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9480000000000001e-06 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231482+0000", 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067510501000000001 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231584+0000", 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010226900000000001 2024-09-16T11:14:21.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231716+0000", 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000131459 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50374 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190457258000002, 2024-09-16T11:14:21.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067897251000000006, 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50374 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164079+0000", 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164080+0000", 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1600000000000001e-07 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164083+0000", 2024-09-16T11:14:21.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164092+0000", 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8060000000000001e-06 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164097+0000", 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8080000000000004e-06 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231753+0000", 2024-09-16T11:14:21.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067656049999999995 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231851+0000", 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8413999999999997e-05 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.231976+0000", 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012493199999999999 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50377 4.12 4:4aef9505:::rbd_data.11b5caf806a8.0000000000000002:10 [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.190016088, 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067709493999999995, 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50377 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164520+0000", 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164521+0000", 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1000000000000001e-06 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164527+0000", 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.874e-06 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164540+0000", 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2792999999999999e-05 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.164611+0000", 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1129000000000003e-05 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232010+0000", 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067399098000000005 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232105+0000", 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4592999999999994e-05 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232230+0000", 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124907 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.425 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50384 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.189385215999998, 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067256723000000004, 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50384 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165151+0000", 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165152+0000", 2024-09-16T11:14:21.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8899999999999998e-07 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165156+0000", 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165168+0000", 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2444e-05 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165175+0000", 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.178e-06 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232255+0000", 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067080579000000001 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232325+0000", 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0230999999999999e-05 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232408+0000", 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2497000000000007e-05 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50387 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.189356934000003, 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067507592000000005, 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.430 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50387 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165179+0000", 2024-09-16T11:14:21.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165180+0000", 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7299999999999996e-07 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165182+0000", 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6190000000000001e-06 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165188+0000", 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0689999999999998e-06 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165191+0000", 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232451+0000", 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067260226000000006 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232552+0000", 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100747 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232687+0000", 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013492199999999999 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50390 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 712704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.434 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.189341706, 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067765773000000001, 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50390 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165195+0000", 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1600000000000001e-07 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165197+0000", 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165203+0000", 2024-09-16T11:14:21.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165205+0000", 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3099999999999999e-06 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232724+0000", 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067519023999999997 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232826+0000", 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000101667 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.232960+0000", 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000134473 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50393 4.12 4:4b40a4e4:::rbd_data.11b5caf806a8.000000000000000a:10 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:21.439 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.189328803999999, 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067968892000000003, 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50393 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165208+0000", 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1600000000000002e-07 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165209+0000", 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.356e-06 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165213+0000", 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2160000000000002e-06 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.165216+0000", 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233022+0000", 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.067805375000000001 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233088+0000", 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6184000000000002e-05 2024-09-16T11:14:21.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.233177+0000", 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8775000000000003e-05 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:21.444 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.861863412, 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:21.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:21.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.451 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:21.452 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:21.349+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:21.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:21.453 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-16T11:14:21.497 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:21.495+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:21.559 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:21.555+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:21.584 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:21.584 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:21.585 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:21.601 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:21.659 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:21.653+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:21.741 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:21.741 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:21.741 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:21.752 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:21.760 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:21.753+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:21.861 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:21.853+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:21.879 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:21.879 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:21.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:10 4.1c 4:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.755317500000004, 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.011183291999999999, 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 10 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.880 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120814+0000", 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120816+0000", 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.278e-06 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120826+0000", 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8840000000000006e-06 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120880+0000", 2024-09-16T11:14:21.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3664e-05 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120891+0000", 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1078e-05 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120966+0000", 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5358000000000006e-05 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:21.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.120974+0000", 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.272e-06 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125014+0000", 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0040399809999999998 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125174+0000", 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000160657 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.125424+0000", 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025006600000000001 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.129365+0000", 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039402209999999998 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131819+0000", 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0024541570000000002 2024-09-16T11:14:21.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131922+0000", 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103016 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:15.131997+0000", 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5660000000000004e-05 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.d 79/78 [PushOp(4:b5f11078:::rbd_data.11b5caf806a8.0000000000000009:head, version: 54'127, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b5f11078:::rbd_data.11b5caf806a8.0000000000000009:head@54'127, size: 1572864, copy_subset: [0~1572864], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.228099+0000", 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.64803224500000001, 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0028792549999999998, 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.228099+0000", 2024-09-16T11:14:21.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.228099+0000", 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.228100+0000", 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1900000000000002e-07 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.229600+0000", 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014994819999999999 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.229625+0000", 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5015999999999999e-05 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.229637+0000", 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2646000000000001e-05 2024-09-16T11:14:21.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.229718+0000", 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1101999999999997e-05 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.229726+0000", 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9230000000000002e-06 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.230979+0000", 2024-09-16T11:14:21.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012523670000000001 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.d 79/78 [PushOp(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head, version: 66'156, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head@66'156, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.276554+0000", 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.59957799599999995, 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027079930000000001, 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.276554+0000", 2024-09-16T11:14:21.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.276554+0000", 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.276555+0000", 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5239999999999999e-06 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.277759+0000", 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00120363 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.277792+0000", 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3003999999999998e-05 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.277804+0000", 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2577e-05 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.277906+0000", 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010129899999999999 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.277920+0000", 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4174e-05 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.279262+0000", 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0013417850000000001 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.d 79/78 [PushOp(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head, version: 66'156, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head@66'156, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.281003+0000", 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.59512913599999995, 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025469730000000001, 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.281003+0000", 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.281003+0000", 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.281004+0000", 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5290000000000001e-06 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.282205+0000", 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001201101 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.282233+0000", 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8215999999999999e-05 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.282246+0000", 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2469999999999999e-05 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.282346+0000", 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010012499999999999 2024-09-16T11:14:21.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.282360+0000", 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3675999999999999e-05 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.283550+0000", 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001189856 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.898 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.d 79/78 [PushOp(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head, version: 66'156, data_included: [2097152~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head@66'156, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2097152, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.285360+0000", 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.59077176499999995, 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0025621960000000001, 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.285360+0000", 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.285360+0000", 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.285362+0000", 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5850000000000001e-06 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.286556+0000", 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001194957 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.286594+0000", 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7586000000000002e-05 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.286610+0000", 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6019e-05 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.286709+0000", 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9351000000000005e-05 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.286722+0000", 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2649e-05 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.287922+0000", 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012000489999999999 2024-09-16T11:14:21.902 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.d 79/78 [PushOp(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head, version: 66'156, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:b711ee6b:::rbd_data.11b5f4eed9b1.0000000000000008:head@66'156, size: 4194304, copy_subset: [0~4194304], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:3145728, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.291746+0000", 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.584385445, 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002933616, 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.291746+0000", 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.291746+0000", 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.291748+0000", 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.64e-06 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.292986+0000", 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012381720000000001 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.293023+0000", 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6804999999999997e-05 2024-09-16T11:14:21.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.293038+0000", 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5522000000000001e-05 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.293161+0000", 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000123075 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.293174+0000", 2024-09-16T11:14:21.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2602000000000001e-05 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.294680+0000", 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0015058000000000001 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop(client.4533.0:94817 4.19 e79/78 4:9e9f3f03:::rbd_header.11b5cb71539e:head v 79'311, mlcod=44'66)", 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.397575+0000", 2024-09-16T11:14:21.907 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.47855636200000001, 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031368250000000002, 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397575+0000", 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397575+0000", 2024-09-16T11:14:21.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397576+0000", 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219999999999999e-06 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397585+0000", 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9549999999999998e-06 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397602+0000", 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7034e-05 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397610+0000", 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1879999999999996e-06 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397668+0000", 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7986999999999999e-05 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.397724+0000", 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6947999999999998e-05 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.400660+0000", 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029351289999999999 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.400712+0000", 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2562000000000003e-05 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9faa7e8a:::rbd_data.11b5624fb08c.0000000000000018:14, version: 62'210, data_included: [0~143884], data_size: 143884, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:9faa7e8a:::rbd_data.11b5624fb08c.0000000000000018:14@62'210, size: 143884, copy_subset: [0~143884], clone_subset: {}, snapset: 14=[]:{14=[14]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:143884, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.530452+0000", 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.34567946700000002, 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002550671, 2024-09-16T11:14:21.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530452+0000", 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530452+0000", 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530453+0000", 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.237e-06 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530551+0000", 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7305000000000004e-05 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530582+0000", 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1056999999999998e-05 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530596+0000", 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4557e-05 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530715+0000", 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118993 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.530730+0000", 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4892000000000001e-05 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.533003+0000", 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022726299999999999 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15, version: 64'243, data_included: [0~65536,81920~114688,212992~114688,344064~114688,475136~245760,737280~114688,868352~114688,999424~114688,1130496~49152], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15@64'243, size: 21718352, copy_subset: [0~21718352], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1179648, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.597045+0000", 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.27908665100000002, 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030157140000000001, 2024-09-16T11:14:21.916 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597045+0000", 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597045+0000", 2024-09-16T11:14:21.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597046+0000", 2024-09-16T11:14:21.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.64e-07 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597654+0000", 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00060841499999999995 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597688+0000", 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3263000000000003e-05 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597703+0000", 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4848000000000001e-05 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597816+0000", 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000113695 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.597828+0000", 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1708999999999999e-05 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.600061+0000", 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0022328199999999999 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15, version: 64'243, data_included: [1179648~65536,1261568~114688,1392640~114688,1523712~114688,1654784~114688,1785856~114688,1916928~114688,2048000~294912], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15@64'243, size: 21718352, copy_subset: [0~21718352], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2342912, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1179648, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.603050+0000", 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.27308144400000001, 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002725482, 2024-09-16T11:14:21.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603050+0000", 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603050+0000", 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603051+0000", 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.158e-06 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603587+0000", 2024-09-16T11:14:21.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053572699999999995 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603615+0000", 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7619000000000001e-05 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603628+0000", 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3654e-05 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603745+0000", 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000116693 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.603754+0000", 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9670000000000007e-06 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.605776+0000", 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020216639999999998 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15, version: 64'243, data_included: [2342912~303104,2662400~81920,2752512~24576,2793472~81920,2883584~24576,2924544~81920,3014656~24576,3055616~81920,3145728~24576,3186688~81920,3276800~24576,3317760~81920,3407872~24576,3448832~81920,3538944~24576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15@64'243, size: 21718352, copy_subset: [0~21718352], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3563520, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2342912, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.609321+0000", 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.26681057400000002, 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023791730000000001, 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.609321+0000", 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.609321+0000", 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.609322+0000", 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.049e-06 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.609876+0000", 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00055370099999999998 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.609908+0000", 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2073999999999998e-05 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.609924+0000", 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5879000000000002e-05 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.610051+0000", 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000127114 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.610059+0000", 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8539999999999997e-06 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.611700+0000", 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001641502 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15, version: 64'243, data_included: [6373376~81920,6504448~81920,6635520~81920,6766592~49152,7236169~23206,7870633~26762,8388608~16384,8413184~90112,8511488~24576,8544256~90112,8642560~24576,8675328~90112,8773632~24576,8806400~90112,8904704~24576,8937472~90112,9035776~24576,9068544~90112,9166848~23760], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15@64'243, size: 21718352, copy_subset: [0~21718352], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:9190608, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:6324224, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.619742+0000", 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.25639014900000001, 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003412647, 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.619742+0000", 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.619742+0000", 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.619743+0000", 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5499999999999996e-07 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.620223+0000", 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048020799999999998 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.620251+0000", 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8265000000000001e-05 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.620266+0000", 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5282e-05 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.620390+0000", 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012327799999999999 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.620401+0000", 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.119e-05 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.623154+0000", 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027534690000000001 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15, version: 64'243, data_included: [9190608~816,9199616~90112,9297920~24576,9330688~90112,9428992~24576,9461760~90112,9560064~24576,9609216~73728,9699328~16384,9740288~73728,9830400~16384,9871360~73728,9961472~11754,10485760~8192,10510336~81920,10608640~16384,10641408~81920,10739712~16384,10772480~81920,10870784~16384,10903552~81920,11001856~16384,11034624~36582], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15@64'243, size: 21718352, copy_subset: [0~21718352], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:11071206, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:9190608, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.625214+0000", 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.25091819900000001, 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0061575349999999996, 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.625214+0000", 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.625214+0000", 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.625215+0000", 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0210000000000001e-06 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.625811+0000", 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000596767 2024-09-16T11:14:21.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.625840+0000", 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8306e-05 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.625856+0000", 2024-09-16T11:14:21.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6483000000000001e-05 2024-09-16T11:14:21.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.626026+0000", 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016966 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.626034+0000", 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3860000000000007e-06 2024-09-16T11:14:21.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.631371+0000", 2024-09-16T11:14:21.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0053369120000000001 2024-09-16T11:14:21.940 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.940 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.941 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.19 79/78 [PushOp(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15, version: 64'243, data_included: [11071206~45338,11132928~16384,11165696~81920,11264000~16384,11296768~81920,11395072~16384,11427840~81920,11526144~16384,11558912~81920,11657216~8192,11689984~49152,11788288~8192,11821056~49152,11919360~8192,11952128~49152,12050432~8192,12083200~49152,12181504~8192,12214272~49152,12312576~8192,14680064~24576,14778368~24576,14811136~24576,14909440~24576,14942208~24576,15040512~24576,15073280~24576,15171584~24576,15204352~24576,15302656~24576,15335424~24576,15433728~24576,21699326~19026], data_size: 1047404, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:9fa1f7b7:::rbd_data.11b579377fe3.0000000000000002:15@64'243, size: 21718352, copy_subset: [0~21718352], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:21718352, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:11071206, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.941 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.633540+0000", 2024-09-16T11:14:21.941 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.24259170299999999, 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0052255230000000001, 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.633540+0000", 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.633540+0000", 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.633541+0000", 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2869999999999999e-06 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.634153+0000", 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000611689 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.634186+0000", 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3067e-05 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.634206+0000", 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9630999999999999e-05 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.634333+0000", 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012718399999999999 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.634343+0000", 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7790000000000007e-06 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.638766+0000", 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004422886 2024-09-16T11:14:21.945 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.4 79/78 [PushOp(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15, version: 64'119, data_included: [0~32768,49152~114688,180224~114688,311296~114688,442368~114688,573440~114688,704512~114688,835584~114688,966656~114688,1097728~98304], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15@64'119, size: 17039360, copy_subset: [0~17039360], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1196032, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.821961+0000", 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.054171030000000002, 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026018090000000001, 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.821961+0000", 2024-09-16T11:14:21.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.821961+0000", 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.821962+0000", 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.187e-06 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.822361+0000", 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000399558 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.822385+0000", 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3485999999999998e-05 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.822395+0000", 2024-09-16T11:14:21.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8800000000000003e-06 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.822521+0000", 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012633800000000001 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.822547+0000", 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.563e-05 2024-09-16T11:14:21.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.824562+0000", 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020157299999999999 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.4 79/78 [PushOp(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15, version: 64'119, data_included: [1196032~16384,1228800~114688,1359872~114688,1490944~114688,1622016~114688,1753088~114688,1884160~114688,2015232~344064], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15@64'119, size: 17039360, copy_subset: [0~17039360], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2359296, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1196032, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.829285+0000", 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.046846962999999998, 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031345069999999999, 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.829285+0000", 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.829285+0000", 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.829286+0000", 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2559999999999999e-06 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.830081+0000", 2024-09-16T11:14:21.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00079495900000000003 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.830111+0000", 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0434999999999999e-05 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.830125+0000", 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3837e-05 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.830249+0000", 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012328799999999999 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.830258+0000", 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0850000000000003e-06 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.832419+0000", 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002161647 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.954 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.4 79/78 [PushOp(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15, version: 64'119, data_included: [2359296~352256,2727936~24576,2760704~81920,2859008~24576,2891776~81920,2990080~24576,3022848~81920,3121152~24576,3153920~81920,3252224~24576,3284992~81920,3383296~24576,3416064~81920,3514368~24576,3547136~32768], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15@64'119, size: 17039360, copy_subset: [0~17039360], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3579904, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:2359296, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.838763+0000", 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.037368782000000003, 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029044689999999998, 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.838763+0000", 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.838763+0000", 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.838764+0000", 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.142e-06 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.839359+0000", 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000594784 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.839382+0000", 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2682999999999999e-05 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.839392+0000", 2024-09-16T11:14:21.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0509999999999999e-05 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.839501+0000", 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000109257 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.839511+0000", 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5219999999999996e-06 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.841667+0000", 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021565709999999999 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.4 79/78 [PushOp(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15, version: 64'119, data_included: [4743168~78348,6340608~49152,6471680~49152,6602752~49152,6733824~49152,6864896~49152,7523813~61152,8388608~81920,8478720~122880,8609792~122880,8740864~122880,8871936~122880,9003008~89876], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15@64'119, size: 17039360, copy_subset: [0~17039360], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:9092884, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:4743168, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.854499+0000", 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.021633111, 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0030141399999999998, 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.959 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.854499+0000", 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.854499+0000", 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.854500+0000", 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4640000000000001e-06 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.854990+0000", 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048949499999999995 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855015+0000", 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5125000000000001e-05 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855027+0000", 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2149e-05 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855138+0000", 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000110809 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855148+0000", 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172e-05 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.857513+0000", 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0023649259999999998 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.4 79/78 [PushOp(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15, version: 64'119, data_included: [9092884~33004,9134080~122880,9265152~122880,9396224~122880,9527296~122880,9658368~40960,9707520~73728,9805824~24576,9838592~73728,9936896~24576,9969664~73728,10067968~24576,10485760~73728,10575872~114452], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15@64'119, size: 17039360, copy_subset: [0~17039360], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:10690324, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:9092884, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.862084+0000", 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.014047921999999999, 2024-09-16T11:14:21.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0042498129999999999, 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862084+0000", 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862084+0000", 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862085+0000", 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.029e-06 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862786+0000", 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00070113399999999996 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862817+0000", 2024-09-16T11:14:21.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0876000000000001e-05 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862831+0000", 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.4435e-05 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862990+0000", 2024-09-16T11:14:21.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000158424 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863000+0000", 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0796e-05 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866334+0000", 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0033331189999999998 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPush(4.4 79/78 [PushOp(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15, version: 64'119, data_included: [10690324~236,10706944~114688,10838016~114688,10969088~114688,11100160~114688,11231232~114688,11362304~114688,11493376~114688,11624448~114688,11755520~81920,11886592~48916], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(4:24d8c815:::rbd_data.11b579377fe3.0000000000000000:15@64'119, size: 17039360, copy_subset: [0~17039360], clone_subset: {}, snapset: 15=[]:{15=[15]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:11935508, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:10690324, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.869832+0000", 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.0062994970000000003, 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029655929999999999, 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869832+0000", 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869832+0000", 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869833+0000", 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.048e-06 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870498+0000", 2024-09-16T11:14:21.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066443399999999999 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870525+0000", 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7367000000000001e-05 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870542+0000", 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7424000000000001e-05 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870644+0000", 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010200300000000001 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870672+0000", 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7681000000000001e-05 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.872798+0000", 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0021256360000000002 2024-09-16T11:14:21.971 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:21.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:21.972 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:21.973 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:21.965+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:21.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:22.074 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:22.069+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:22.086 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:14:22.108 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:22.108 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:22.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.108 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4.c0fcf979 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:22.108 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.030184539999999999, 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.030207405, 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:14:22.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:14:22.113 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:22.117 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:22.175 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:22.169+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:22.213 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:22.213 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:22.213 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:22.213 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:22.213 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:22.219 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:22.276 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:22.271+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:22.322 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:22.322 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:22.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:22.322 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:22.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49775 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1286144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.185020045000002, 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050091745, 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49775 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:22.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:22.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133156+0000", 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1179999999999999e-06 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133159+0000", 2024-09-16T11:14:22.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:22.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133178+0000", 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8692e-05 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.182940+0000", 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049762038000000001 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183007+0000", 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7082000000000006e-05 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183247+0000", 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239581 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184995108999999, 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.329 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:14:22.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184861366, 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:22.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:22.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:22.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184842977999999, 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:22.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:22.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184542221000001, 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:22.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184512515999998, 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.348 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:22.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:22.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:22.352 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184499117999998, 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184485314, 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.357 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:22.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:22.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.184110314999998, 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:22.361 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:22.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50035 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.170582007, 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049942324000000003, 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50035 2024-09-16T11:14:22.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147593+0000", 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147595+0000", 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9769999999999999e-06 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147601+0000", 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147605+0000", 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197156+0000", 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049550679 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197231+0000", 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5853999999999996e-05 2024-09-16T11:14:22.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197535+0000", 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000303861 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.170566284000003, 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.370 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:22.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.170544710000001, 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:22.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.378 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.170528210999997, 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.379 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:22.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.170512135000003, 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:22.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.170128560999999, 2024-09-16T11:14:22.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:22.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:22.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50292 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.158065663000002, 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050216150000000001, 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50292 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.392 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160110+0000", 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160117+0000", 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.362e-06 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160124+0000", 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:14:22.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160126+0000", 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210014+0000", 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049887881000000002 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210108+0000", 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3356000000000005e-05 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210325+0000", 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021782400000000001 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.158046384000002, 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.397 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:22.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.158038183000002, 2024-09-16T11:14:22.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:22.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.405 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.158030160999999, 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:22.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:22.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.4745512820000002, 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:22.410 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:22.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:14:22.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:22.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:14:22.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:14:22.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:22.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:22.416 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:22.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:22.418 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:22.415+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:22.519 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:22.513+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101618 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067354+0000", 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464275469, 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46431577899999998, 2024-09-16T11:14:22.561 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101618 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067354+0000", 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067354+0000", 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067355+0000", 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7400000000000003e-07 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067357+0000", 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3630000000000001e-06 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067364+0000", 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0600000000000002e-06 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067369+0000", 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1999999999999996e-06 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076704+0000", 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093359189999999998 2024-09-16T11:14:22.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076708+0000", 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.597e-06 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101741 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068143+0000", 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463486338, 2024-09-16T11:14:22.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46363885399999999, 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101741 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068143+0000", 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068143+0000", 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068144+0000", 2024-09-16T11:14:22.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068146+0000", 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1320000000000001e-06 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068157+0000", 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0617e-05 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068161+0000", 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1659999999999996e-06 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077599+0000", 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094378529999999995 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077602+0000", 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7790000000000002e-06 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101814 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068963+0000", 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46266695600000002, 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46291601999999998, 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.570 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101814 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068963+0000", 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068963+0000", 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068963+0000", 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068965+0000", 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.99e-06 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068977+0000", 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1928e-05 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068981+0000", 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9360000000000001e-06 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078544+0000", 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095632800000000004 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078548+0000", 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6669999999999998e-06 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101859 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.574 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069718+0000", 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46191214600000002, 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46225406499999999, 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101859 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069718+0000", 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069718+0000", 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069718+0000", 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069720+0000", 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7099999999999999e-06 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069724+0000", 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8850000000000001e-06 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069727+0000", 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5209999999999998e-06 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079512+0000", 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097845129999999999 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079515+0000", 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6330000000000002e-06 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102097 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071318+0000", 2024-09-16T11:14:22.578 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46031127999999999, 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.460746926, 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102097 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071318+0000", 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071318+0000", 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071319+0000", 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2900000000000003e-07 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071322+0000", 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3879999999999998e-06 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071328+0000", 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5220000000000004e-06 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071332+0000", 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080486+0000", 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091545419999999999 2024-09-16T11:14:22.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080492+0000", 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5720000000000002e-06 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102371 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072446+0000", 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45918393800000001, 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.45971210000000001, 2024-09-16T11:14:22.582 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102371 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072446+0000", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072446+0000", 2024-09-16T11:14:22.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072446+0000", 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.99e-07 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072449+0000", 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8789999999999999e-06 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072472+0000", 2024-09-16T11:14:22.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2359999999999999e-05 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072478+0000", 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.28e-06 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081592+0000", 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091135850000000004 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081595+0000", 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.596e-06 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102604 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073103+0000", 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45852685100000001, 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.45914816400000003, 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.586 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102604 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073103+0000", 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073103+0000", 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073103+0000", 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073106+0000", 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6410000000000002e-06 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073112+0000", 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2689999999999996e-06 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073117+0000", 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0370000000000002e-06 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082445+0000", 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093273339999999996 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082448+0000", 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7249999999999999e-06 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102657 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073866+0000", 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45776335200000001, 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.45848008600000001, 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102657 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073866+0000", 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073866+0000", 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073867+0000", 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4199999999999995e-07 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073884+0000", 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6569e-05 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073892+0000", 2024-09-16T11:14:22.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.833e-06 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073899+0000", 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8469999999999998e-06 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082979+0000", 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090799820000000003 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082985+0000", 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4979999999999997e-06 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101619 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067373+0000", 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46425692299999999, 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.465065589, 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101619 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067373+0000", 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067373+0000", 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067373+0000", 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9599999999999999e-07 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067376+0000", 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2409999999999998e-06 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067381+0000", 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2530000000000004e-06 2024-09-16T11:14:22.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067383+0000", 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4700000000000001e-06 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076727+0000", 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093440539999999992 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076731+0000", 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101743 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068165+0000", 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463464714, 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46436504099999998, 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101743 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068165+0000", 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068165+0000", 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068165+0000", 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9999999999999998e-07 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068168+0000", 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4140000000000001e-06 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068178+0000", 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0409e-05 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068183+0000", 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4950000000000002e-06 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094450179999999995 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077631+0000", 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.602 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101816 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068993+0000", 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46263713299999998, 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46367029199999998, 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101816 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068993+0000", 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068993+0000", 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068993+0000", 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8900000000000001e-07 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068995+0000", 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7039999999999999e-06 2024-09-16T11:14:22.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068998+0000", 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.535e-06 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069001+0000", 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.48e-06 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078572+0000", 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095714949999999993 2024-09-16T11:14:22.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078576+0000", 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0010000000000003e-06 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101861 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069764+0000", 2024-09-16T11:14:22.607 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.461865576, 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46303783399999998, 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101861 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069764+0000", 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069764+0000", 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069764+0000", 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1699999999999999e-07 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069766+0000", 2024-09-16T11:14:22.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.779e-06 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069770+0000", 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9689999999999996e-06 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069774+0000", 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7069999999999998e-06 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079541+0000", 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009766828 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079544+0000", 2024-09-16T11:14:22.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4079999999999998e-06 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102098 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071354+0000", 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.460275355, 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46153983799999998, 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102098 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071354+0000", 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071354+0000", 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071355+0000", 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1200000000000002e-07 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071358+0000", 2024-09-16T11:14:22.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7319999999999998e-06 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071372+0000", 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3789e-05 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071377+0000", 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9740000000000001e-06 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080518+0000", 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091412090000000008 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080521+0000", 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4170000000000001e-06 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102372 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072483+0000", 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45914648299999999, 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46050267, 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102372 2024-09-16T11:14:22.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072483+0000", 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072483+0000", 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072484+0000", 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.37e-07 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072486+0000", 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6469999999999998e-06 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072493+0000", 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2009999999999996e-06 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072497+0000", 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8809999999999998e-06 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081620+0000", 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091239370000000004 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081624+0000", 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4470000000000002e-06 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.620 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102659 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073906+0000", 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45772405300000002, 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.45917746199999998, 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102659 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073906+0000", 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073906+0000", 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073906+0000", 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2699999999999999e-07 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073909+0000", 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3700000000000002e-06 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073915+0000", 2024-09-16T11:14:22.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073921+0000", 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9070000000000004e-06 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083011+0000", 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090905069999999994 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083014+0000", 2024-09-16T11:14:22.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3079999999999999e-06 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101621 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067418+0000", 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464211241, 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46575577800000001, 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101621 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067418+0000", 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067418+0000", 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067419+0000", 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-07 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067421+0000", 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067433+0000", 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1643e-05 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067438+0000", 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0860000000000001e-06 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076750+0000", 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009312341 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076754+0000", 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.461e-06 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101745 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068187+0000", 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46344246500000003, 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46507796000000001, 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101745 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068187+0000", 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068187+0000", 2024-09-16T11:14:22.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068188+0000", 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8500000000000002e-07 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068190+0000", 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068200+0000", 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0514e-05 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068204+0000", 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.771e-06 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077657+0000", 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094523739999999995 2024-09-16T11:14:22.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077660+0000", 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6339999999999999e-06 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.633 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101818 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069022+0000", 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.462607347, 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46433391699999998, 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.635 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101818 2024-09-16T11:14:22.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.635 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069022+0000", 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069022+0000", 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069023+0000", 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5400000000000001e-07 2024-09-16T11:14:22.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069029+0000", 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7080000000000002e-06 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069040+0000", 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1454000000000001e-05 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069046+0000", 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5400000000000003e-06 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078604+0000", 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095586890000000004 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078608+0000", 2024-09-16T11:14:22.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101863 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069824+0000", 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46180590500000002, 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46362345599999999, 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101863 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069824+0000", 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069824+0000", 2024-09-16T11:14:22.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069825+0000", 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7000000000000003e-07 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069829+0000", 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6539999999999998e-06 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069841+0000", 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2021999999999999e-05 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069848+0000", 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8580000000000002e-06 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079567+0000", 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097190130000000003 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079571+0000", 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6049999999999998e-06 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102100 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071391+0000", 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46023885199999998, 2024-09-16T11:14:22.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.462170735, 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102100 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071391+0000", 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071391+0000", 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071392+0000", 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8299999999999995e-07 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071395+0000", 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8609999999999998e-06 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071401+0000", 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3430000000000001e-06 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071405+0000", 2024-09-16T11:14:22.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.873e-06 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080547+0000", 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091417689999999992 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080552+0000", 2024-09-16T11:14:22.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.694e-06 2024-09-16T11:14:22.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.648 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.649 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102373 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.649 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072501+0000", 2024-09-16T11:14:22.649 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45912908099999999, 2024-09-16T11:14:22.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.461152852, 2024-09-16T11:14:22.649 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.649 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.650 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102373 2024-09-16T11:14:22.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072501+0000", 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072501+0000", 2024-09-16T11:14:22.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072501+0000", 2024-09-16T11:14:22.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:14:22.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072506+0000", 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5639999999999998e-06 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072511+0000", 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.925e-06 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072515+0000", 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.472e-06 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081651+0000", 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091360720000000003 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081655+0000", 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6500000000000002e-06 2024-09-16T11:14:22.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102606 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073135+0000", 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45849448700000001, 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46060883600000002, 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102606 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073135+0000", 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073135+0000", 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073136+0000", 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:22.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073138+0000", 2024-09-16T11:14:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0209999999999998e-06 2024-09-16T11:14:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073144+0000", 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9540000000000001e-06 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073147+0000", 2024-09-16T11:14:22.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6770000000000001e-06 2024-09-16T11:14:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082475+0000", 2024-09-16T11:14:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093279590000000003 2024-09-16T11:14:22.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082479+0000", 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102661 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073934+0000", 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45769558599999999, 2024-09-16T11:14:22.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.45990093599999998, 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102661 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073934+0000", 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073934+0000", 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073935+0000", 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0699999999999997e-07 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073937+0000", 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.221e-06 2024-09-16T11:14:22.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073944+0000", 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3379999999999999e-06 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073949+0000", 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4299999999999999e-06 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083043+0000", 2024-09-16T11:14:22.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090940180000000006 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083046+0000", 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7220000000000001e-06 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101622 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067443+0000", 2024-09-16T11:14:22.665 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464186812, 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46648232299999998, 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101622 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067443+0000", 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067443+0000", 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067443+0000", 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4700000000000002e-07 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067445+0000", 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0700000000000001e-06 2024-09-16T11:14:22.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067452+0000", 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1990000000000003e-06 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067455+0000", 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076774+0000", 2024-09-16T11:14:22.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093181559999999993 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076777+0000", 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3340000000000002e-06 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.669 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101747 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068215+0000", 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46341449499999998, 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46580138799999998, 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101747 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068215+0000", 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068215+0000", 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068216+0000", 2024-09-16T11:14:22.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8700000000000005e-07 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068218+0000", 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2749999999999998e-06 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068226+0000", 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3339999999999996e-06 2024-09-16T11:14:22.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068230+0000", 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9840000000000003e-06 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077687+0000", 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094577729999999992 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101820 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069064+0000", 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46256540200000001, 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46504320900000001, 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.674 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101820 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069064+0000", 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069064+0000", 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069065+0000", 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7700000000000003e-07 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069068+0000", 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.311e-06 2024-09-16T11:14:22.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069072+0000", 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3699999999999997e-06 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069077+0000", 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9370000000000003e-06 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078633+0000", 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095563349999999991 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078637+0000", 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.416e-06 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.678 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101865 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069883+0000", 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46174695100000002, 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.464315107, 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101865 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069883+0000", 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069883+0000", 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069884+0000", 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069887+0000", 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9919999999999999e-06 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069894+0000", 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2099999999999996e-06 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069898+0000", 2024-09-16T11:14:22.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3039999999999998e-06 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079595+0000", 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096967640000000001 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079598+0000", 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3459999999999998e-06 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102101 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071520+0000", 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46010962100000002, 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46276900100000001, 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102101 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071520+0000", 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071520+0000", 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071521+0000", 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6899999999999996e-07 2024-09-16T11:14:22.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071524+0000", 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1700000000000001e-06 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071534+0000", 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4949999999999997e-06 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071539+0000", 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0329999999999999e-06 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080711+0000", 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091722690000000003 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080719+0000", 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3680000000000002e-06 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102375 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072519+0000", 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.459110403, 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46194264299999999, 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102375 2024-09-16T11:14:22.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072519+0000", 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072519+0000", 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072520+0000", 2024-09-16T11:14:22.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.96e-07 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072532+0000", 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2492e-05 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072540+0000", 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6420000000000007e-06 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072544+0000", 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081679+0000", 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091347860000000006 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081682+0000", 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:22.690 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102663 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073955+0000", 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45767495200000002, 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46059959, 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.691 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102663 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073955+0000", 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073955+0000", 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073955+0000", 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073957+0000", 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1289999999999999e-06 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073964+0000", 2024-09-16T11:14:22.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2750000000000001e-06 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073968+0000", 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.138e-06 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083072+0000", 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091037759999999992 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083075+0000", 2024-09-16T11:14:22.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3909999999999998e-06 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101623 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067460+0000", 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46417011400000002, 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46718985299999999, 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101623 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067460+0000", 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067460+0000", 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067460+0000", 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4499999999999997e-07 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067462+0000", 2024-09-16T11:14:22.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.922e-06 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067469+0000", 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7950000000000002e-06 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067473+0000", 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.675e-06 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076797+0000", 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093245180000000004 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076800+0000", 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4139999999999998e-06 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101771 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068256+0000", 2024-09-16T11:14:22.699 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463374222, 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46648506899999997, 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101771 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068256+0000", 2024-09-16T11:14:22.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068256+0000", 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068256+0000", 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068258+0000", 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6780000000000001e-06 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068262+0000", 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:22.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068264+0000", 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6790000000000001e-06 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077715+0000", 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094505230000000006 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077719+0000", 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8240000000000001e-06 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101822 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069089+0000", 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46254102000000002, 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46574870499999999, 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.704 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101822 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069089+0000", 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069089+0000", 2024-09-16T11:14:22.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069089+0000", 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069091+0000", 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.052e-06 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069097+0000", 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.789e-06 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069101+0000", 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1389999999999997e-06 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078663+0000", 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095615179999999998 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078666+0000", 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.749e-06 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102057 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070714+0000", 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46091591999999998, 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46421480700000001, 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.708 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102057 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070714+0000", 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070714+0000", 2024-09-16T11:14:22.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070715+0000", 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1800000000000005e-07 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070718+0000", 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.36e-06 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070728+0000", 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9329999999999996e-06 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070733+0000", 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8289999999999998e-06 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079625+0000", 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088925470000000006 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079629+0000", 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.72e-06 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102311 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071543+0000", 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46008656599999997, 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46348034399999999, 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.712 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102311 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071543+0000", 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071543+0000", 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071544+0000", 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200000000000004e-07 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071546+0000", 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.849e-06 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071551+0000", 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.711e-06 2024-09-16T11:14:22.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071555+0000", 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0149999999999997e-06 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080749+0000", 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091936009999999992 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080752+0000", 2024-09-16T11:14:22.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102501 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072548+0000", 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45908151000000003, 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46256935300000002, 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102501 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072548+0000", 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072548+0000", 2024-09-16T11:14:22.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072549+0000", 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072551+0000", 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072557+0000", 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7209999999999999e-06 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072560+0000", 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081708+0000", 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091478080000000003 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081712+0000", 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6940000000000001e-06 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102608 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073167+0000", 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45846315100000001, 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46204194500000001, 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102608 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073167+0000", 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073167+0000", 2024-09-16T11:14:22.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073167+0000", 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4700000000000002e-07 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073169+0000", 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073175+0000", 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0769999999999996e-06 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073179+0000", 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6550000000000002e-06 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082507+0000", 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093278489999999992 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082510+0000", 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5599999999999998e-06 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102665 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073974+0000", 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.457655698, 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46132867700000002, 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102665 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073974+0000", 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073974+0000", 2024-09-16T11:14:22.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073975+0000", 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0699999999999996e-07 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073977+0000", 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2419999999999999e-06 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073983+0000", 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2770000000000002e-06 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073987+0000", 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7390000000000001e-06 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083100+0000", 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091131530000000006 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083104+0000", 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101625 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067485+0000", 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46414444100000002, 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46790922200000001, 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.728 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101625 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067485+0000", 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067485+0000", 2024-09-16T11:14:22.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067486+0000", 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7199999999999998e-07 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067499+0000", 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3118e-05 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067505+0000", 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.088e-06 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067509+0000", 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0570000000000003e-06 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076820+0000", 2024-09-16T11:14:22.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093103639999999998 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076823+0000", 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3919999999999999e-06 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101773 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068285+0000", 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46334509899999998, 2024-09-16T11:14:22.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46720427399999997, 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101773 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068285+0000", 2024-09-16T11:14:22.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068285+0000", 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068285+0000", 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5199999999999996e-07 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068287+0000", 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2060000000000001e-06 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068294+0000", 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0390000000000001e-06 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068298+0000", 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0679999999999998e-06 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077743+0000", 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009445663 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077747+0000", 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3079999999999999e-06 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101825 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069105+0000", 2024-09-16T11:14:22.736 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46252451900000002, 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46647483200000001, 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101825 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069105+0000", 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069105+0000", 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069106+0000", 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069108+0000", 2024-09-16T11:14:22.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1500000000000002e-06 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069115+0000", 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3220000000000004e-06 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069118+0000", 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8219999999999998e-06 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078696+0000", 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095778210000000003 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078699+0000", 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4180000000000001e-06 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102059 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.740 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070754+0000", 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46087578400000001, 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46492102899999999, 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102059 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070754+0000", 2024-09-16T11:14:22.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070754+0000", 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070755+0000", 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7199999999999999e-07 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070756+0000", 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.747e-06 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070760+0000", 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1960000000000001e-06 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070763+0000", 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6249999999999999e-06 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079653+0000", 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088898320000000003 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079656+0000", 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3119999999999998e-06 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.744 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102313 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071573+0000", 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46005695400000002, 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.464197313, 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102313 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071573+0000", 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071573+0000", 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071574+0000", 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4700000000000001e-07 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071576+0000", 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.413e-06 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071671+0000", 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4833999999999994e-05 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071678+0000", 2024-09-16T11:14:22.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0530000000000001e-06 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080781+0000", 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091028830000000008 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080784+0000", 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102502 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072564+0000", 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45906586100000002, 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46329768500000001, 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102502 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072564+0000", 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072564+0000", 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072564+0000", 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4499999999999998e-07 2024-09-16T11:14:22.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072566+0000", 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072572+0000", 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072575+0000", 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6370000000000001e-06 2024-09-16T11:14:22.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081735+0000", 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091594880000000004 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081738+0000", 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4120000000000001e-06 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45763867000000003, 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46196485199999998, 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:22.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101627 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067516+0000", 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464113887, 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46853099399999998, 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101627 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067516+0000", 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067516+0000", 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067516+0000", 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0999999999999998e-07 2024-09-16T11:14:22.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067518+0000", 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.041e-06 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067525+0000", 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0220000000000001e-06 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067529+0000", 2024-09-16T11:14:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2969999999999997e-06 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076843+0000", 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093141019999999995 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076847+0000", 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.636e-06 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101775 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068302+0000", 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463328024, 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46784083599999998, 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101775 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068302+0000", 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068302+0000", 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068302+0000", 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:14:22.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068304+0000", 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1950000000000002e-06 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068316+0000", 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2057e-05 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068320+0000", 2024-09-16T11:14:22.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8240000000000001e-06 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077780+0000", 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094599799999999998 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077785+0000", 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8940000000000001e-06 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101827 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069131+0000", 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46249913500000001, 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46710259500000001, 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101827 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069131+0000", 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069131+0000", 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069131+0000", 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999998e-07 2024-09-16T11:14:22.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069133+0000", 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1289999999999999e-06 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069139+0000", 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0179999999999998e-06 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069143+0000", 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9369999999999997e-06 2024-09-16T11:14:22.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078733+0000", 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095895400000000006 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078736+0000", 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.768 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102061 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070773+0000", 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46085681899999997, 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46555818700000001, 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102061 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070773+0000", 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070773+0000", 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070773+0000", 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.13e-07 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070787+0000", 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3621999999999999e-05 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070792+0000", 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.515e-06 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070794+0000", 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6570000000000001e-06 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079684+0000", 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0088900679999999992 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079688+0000", 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7500000000000001e-06 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102316 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071684+0000", 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45994579899999999, 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46474200500000001, 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102316 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071684+0000", 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071684+0000", 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071685+0000", 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8400000000000002e-07 2024-09-16T11:14:22.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071688+0000", 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1190000000000001e-06 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071696+0000", 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6909999999999997e-06 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071700+0000", 2024-09-16T11:14:22.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8630000000000001e-06 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080809+0000", 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091096200000000006 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080813+0000", 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6169999999999998e-06 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102503 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072581+0000", 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45904851600000002, 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46393639399999997, 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102503 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072581+0000", 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072581+0000", 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072582+0000", 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072584+0000", 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.092e-06 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072590+0000", 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072594+0000", 2024-09-16T11:14:22.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.754e-06 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081767+0000", 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091737950000000002 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081771+0000", 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6839999999999998e-06 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102610 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073199+0000", 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45843046399999998, 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46341249600000001, 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102610 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073199+0000", 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073199+0000", 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073200+0000", 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0600000000000001e-07 2024-09-16T11:14:22.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073202+0000", 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2510000000000001e-06 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073208+0000", 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2949999999999999e-06 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073212+0000", 2024-09-16T11:14:22.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.754e-06 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082537+0000", 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093253859999999997 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082541+0000", 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.457619304, 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46269231100000002, 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.785 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:22.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101629 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067549+0000", 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46408117900000001, 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46924904699999997, 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.789 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101629 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067549+0000", 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067549+0000", 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067549+0000", 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7100000000000001e-07 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067552+0000", 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3010000000000001e-06 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067557+0000", 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9510000000000003e-06 2024-09-16T11:14:22.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067562+0000", 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1239999999999998e-06 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076866+0000", 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093044449999999997 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076869+0000", 2024-09-16T11:14:22.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3979999999999999e-06 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101777 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068332+0000", 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46329775099999998, 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46855635699999998, 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101777 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068332+0000", 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068332+0000", 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068332+0000", 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0600000000000001e-07 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068334+0000", 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.483e-06 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068338+0000", 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3599999999999998e-06 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068341+0000", 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3860000000000001e-06 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077812+0000", 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094713220000000008 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077816+0000", 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6040000000000003e-06 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101828 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069156+0000", 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46247373800000002, 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46782704600000002, 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101828 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069156+0000", 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069156+0000", 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069157+0000", 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4700000000000001e-07 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069159+0000", 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2220000000000001e-06 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069165+0000", 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1009999999999997e-06 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069169+0000", 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2579999999999997e-06 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078760+0000", 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095912440000000005 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078764+0000", 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:14:22.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102064 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070797+0000", 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46083262000000003, 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46640918100000001, 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.801 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102064 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070797+0000", 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070797+0000", 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070798+0000", 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3099999999999998e-07 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070799+0000", 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2559999999999999e-06 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070802+0000", 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0070000000000002e-06 2024-09-16T11:14:22.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070806+0000", 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999996e-06 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079712+0000", 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089063719999999992 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079716+0000", 2024-09-16T11:14:22.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3419999999999999e-06 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102318 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071716+0000", 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45991349300000001, 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46558163600000002, 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.805 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102318 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071716+0000", 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071716+0000", 2024-09-16T11:14:22.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071717+0000", 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4499999999999997e-07 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071719+0000", 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.446e-06 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071727+0000", 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6299999999999998e-06 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071731+0000", 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9310000000000001e-06 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080840+0000", 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091087430000000007 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080843+0000", 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4750000000000002e-06 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102504 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072597+0000", 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45903230099999998, 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46479608500000003, 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102504 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072597+0000", 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072597+0000", 2024-09-16T11:14:22.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072598+0000", 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.08e-07 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072600+0000", 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072606+0000", 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.874e-06 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072610+0000", 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081798+0000", 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091884659999999993 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081801+0000", 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3759999999999999e-06 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102611 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073225+0000", 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45840515799999998, 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46426034999999999, 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.813 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102611 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073225+0000", 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073225+0000", 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073225+0000", 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073227+0000", 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073234+0000", 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5679999999999996e-06 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073238+0000", 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7190000000000001e-06 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082567+0000", 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093287869999999998 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082571+0000", 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.704e-06 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45757854199999998, 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46354625900000002, 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.817 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:22.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101630 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067567+0000", 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46406241799999998, 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47012325700000002, 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.821 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101630 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067567+0000", 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067567+0000", 2024-09-16T11:14:22.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067568+0000", 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1399999999999997e-07 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067570+0000", 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.03e-06 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067576+0000", 2024-09-16T11:14:22.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1469999999999998e-06 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067580+0000", 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8029999999999998e-06 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076890+0000", 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093104060000000002 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076894+0000", 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.596e-06 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101779 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068361+0000", 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46326849799999997, 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46942341500000001, 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101779 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068361+0000", 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068361+0000", 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068362+0000", 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499999999999998e-07 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068364+0000", 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.283e-06 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068370+0000", 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8980000000000001e-06 2024-09-16T11:14:22.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068374+0000", 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0339999999999998e-06 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077842+0000", 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009468275 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077846+0000", 2024-09-16T11:14:22.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.54e-06 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101829 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069182+0000", 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46244780099999999, 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.468697066, 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101829 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069182+0000", 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069182+0000", 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069182+0000", 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069185+0000", 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4119999999999999e-06 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069191+0000", 2024-09-16T11:14:22.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2049999999999999e-06 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069195+0000", 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0749999999999999e-06 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078792+0000", 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095971119999999997 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078797+0000", 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4349999999999999e-06 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102068 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070808+0000", 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46082141300000001, 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46716181899999998, 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.833 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102068 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070808+0000", 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070808+0000", 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070809+0000", 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070810+0000", 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3349999999999999e-06 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070813+0000", 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9359999999999999e-06 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070815+0000", 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079744+0000", 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089284710000000003 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079748+0000", 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.839e-06 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102320 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071753+0000", 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45987697399999999, 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.466312485, 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.837 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102320 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071753+0000", 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071753+0000", 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071754+0000", 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9800000000000002e-07 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071756+0000", 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.362e-06 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071763+0000", 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8129999999999998e-06 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071767+0000", 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1470000000000003e-06 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080871+0000", 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091043259999999994 2024-09-16T11:14:22.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080875+0000", 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102505 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072614+0000", 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45901609500000001, 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46554259399999998, 2024-09-16T11:14:22.841 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102505 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072614+0000", 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072614+0000", 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072614+0000", 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7899999999999999e-07 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072616+0000", 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2199999999999999e-06 2024-09-16T11:14:22.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072622+0000", 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7960000000000001e-06 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072626+0000", 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.619e-06 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081834+0000", 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092083449999999997 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081838+0000", 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6720000000000002e-06 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45754774399999998, 2024-09-16T11:14:22.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46416945799999998, 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:22.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:22.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101632 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067606+0000", 2024-09-16T11:14:22.849 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46402406600000001, 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47073725100000002, 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101632 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067606+0000", 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067606+0000", 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067606+0000", 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.61e-07 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067609+0000", 2024-09-16T11:14:22.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3659999999999999e-06 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067615+0000", 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0979999999999999e-06 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067620+0000", 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0590000000000002e-06 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076922+0000", 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093024839999999998 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076926+0000", 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.416e-06 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101781 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068378+0000", 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46325154400000002, 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47005872100000001, 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101781 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068378+0000", 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068378+0000", 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068379+0000", 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068381+0000", 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1579999999999999e-06 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068387+0000", 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3820000000000001e-06 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068396+0000", 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9369999999999993e-06 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077869+0000", 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094734390000000002 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077873+0000", 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101830 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069210+0000", 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46241935499999998, 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46932363300000002, 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101830 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069210+0000", 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069210+0000", 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069211+0000", 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.54e-07 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069213+0000", 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0059999999999999e-06 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069225+0000", 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1272e-05 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069229+0000", 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1649999999999999e-06 2024-09-16T11:14:22.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078825+0000", 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095960139999999999 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078828+0000", 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4970000000000001e-06 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102070 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070817+0000", 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46081239499999999, 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46780798499999998, 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102070 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070817+0000", 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070817+0000", 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070818+0000", 2024-09-16T11:14:22.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070819+0000", 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.204e-06 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070822+0000", 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.977e-06 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070824+0000", 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2129999999999998e-06 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079772+0000", 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089476839999999992 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079775+0000", 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.332e-06 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102322 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071771+0000", 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.459858247, 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46694804400000001, 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102322 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.866 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071771+0000", 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071771+0000", 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071772+0000", 2024-09-16T11:14:22.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6000000000000003e-07 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071774+0000", 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2309999999999999e-06 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071780+0000", 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8340000000000004e-06 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071784+0000", 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00911643 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080904+0000", 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3620000000000002e-06 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102506 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072630+0000", 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45900017999999998, 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.466180859, 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102506 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.870 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072630+0000", 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072630+0000", 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072630+0000", 2024-09-16T11:14:22.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072632+0000", 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.903e-06 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072638+0000", 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.66e-06 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072645+0000", 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8269999999999998e-06 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081865+0000", 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092191770000000003 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081868+0000", 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5779999999999999e-06 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.873 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102613 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073319+0000", 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.458310932, 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46558855599999999, 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102613 2024-09-16T11:14:22.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073319+0000", 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073319+0000", 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073320+0000", 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073323+0000", 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9799999999999998e-06 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073330+0000", 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9090000000000003e-06 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073334+0000", 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.425e-06 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082597+0000", 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092630869999999997 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082600+0000", 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3809999999999999e-06 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45752915599999999, 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46489741099999998, 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:22.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.881 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101558 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066718+0000", 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46491163600000002, 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472374083, 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.882 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101558 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066718+0000", 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066718+0000", 2024-09-16T11:14:22.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066719+0000", 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4399999999999998e-07 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066722+0000", 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.723e-06 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066729+0000", 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7409999999999992e-06 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066735+0000", 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0950000000000003e-06 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076023+0000", 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009288546 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076028+0000", 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8010000000000003e-06 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101633 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067624+0000", 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46400576500000001, 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47156379599999998, 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101633 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067624+0000", 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067624+0000", 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067624+0000", 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4999999999999998e-07 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067627+0000", 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1119999999999999e-06 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067633+0000", 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8059999999999997e-06 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067640+0000", 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076949+0000", 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093086520000000006 2024-09-16T11:14:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076953+0000", 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7280000000000001e-06 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101783 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068426+0000", 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46320382700000001, 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47085268699999999, 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.890 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101783 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068426+0000", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068426+0000", 2024-09-16T11:14:22.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068427+0000", 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0500000000000003e-07 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068439+0000", 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2343e-05 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068445+0000", 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8780000000000003e-06 2024-09-16T11:14:22.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068449+0000", 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1880000000000004e-06 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077911+0000", 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094619679999999994 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:22.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9200000000000004e-06 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101831 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069251+0000", 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.462378811, 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47012194499999999, 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101831 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069251+0000", 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069251+0000", 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069251+0000", 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.72e-07 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069253+0000", 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.148e-06 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069264+0000", 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0171e-05 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069266+0000", 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6230000000000001e-06 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078856+0000", 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095899120000000008 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078860+0000", 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.743e-06 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102072 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070837+0000", 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46079235600000001, 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46862556500000002, 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102072 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.898 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070837+0000", 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070837+0000", 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070838+0000", 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.82e-07 2024-09-16T11:14:22.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070839+0000", 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.567e-06 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070844+0000", 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3449999999999999e-06 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070846+0000", 2024-09-16T11:14:22.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.294e-06 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079802+0000", 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089560479999999994 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079806+0000", 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102325 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071788+0000", 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45984154900000002, 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46776874299999999, 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102325 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.902 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071788+0000", 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071788+0000", 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071789+0000", 2024-09-16T11:14:22.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.89e-07 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071790+0000", 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.849e-06 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071797+0000", 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3210000000000001e-06 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071801+0000", 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8539999999999999e-06 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080931+0000", 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091305919999999999 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080935+0000", 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5870000000000001e-06 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.905 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102507 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072650+0000", 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45897969900000002, 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46699762, 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102507 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072650+0000", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072650+0000", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072650+0000", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4299999999999999e-07 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072652+0000", 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0420000000000001e-06 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072659+0000", 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0929999999999999e-06 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072662+0000", 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.512e-06 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081898+0000", 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092356399999999998 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081901+0000", 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5499999999999999e-06 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:22.909 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.457511104, 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46562343699999997, 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:22.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101560 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066742+0000", 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464887257, 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.473093441, 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101560 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066742+0000", 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066742+0000", 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066743+0000", 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.68e-07 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066746+0000", 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4990000000000001e-06 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066758+0000", 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2024e-05 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066776+0000", 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8105999999999999e-05 2024-09-16T11:14:22.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076146+0000", 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093699409999999997 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076151+0000", 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4709999999999998e-06 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101635 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067651+0000", 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46397918100000002, 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472276107, 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101635 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067651+0000", 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067651+0000", 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067651+0000", 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1e-07 2024-09-16T11:14:22.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067652+0000", 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4899999999999999e-06 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067656+0000", 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4769999999999999e-06 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067659+0000", 2024-09-16T11:14:22.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6119999999999999e-06 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093185179999999996 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076981+0000", 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101784 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068455+0000", 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46317422499999999, 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47156594600000001, 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101784 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068455+0000", 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068455+0000", 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068456+0000", 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8500000000000001e-07 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068458+0000", 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2670000000000001e-06 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068463+0000", 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.369e-06 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068467+0000", 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0729999999999998e-06 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077946+0000", 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094787589999999998 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077949+0000", 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3359999999999999e-06 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101832 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.925 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069269+0000", 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46236057400000002, 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47084270900000003, 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101832 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069269+0000", 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069269+0000", 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069270+0000", 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0100000000000005e-07 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069272+0000", 2024-09-16T11:14:22.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1660000000000001e-06 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069278+0000", 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8549999999999998e-06 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069282+0000", 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0579999999999999e-06 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078886+0000", 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096046280000000005 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078890+0000", 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.461e-06 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102073 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070848+0000", 2024-09-16T11:14:22.929 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.460781365, 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46935757500000003, 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102073 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070848+0000", 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070848+0000", 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070849+0000", 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6e-07 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070850+0000", 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2699999999999999e-06 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070853+0000", 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0469999999999998e-06 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070855+0000", 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0710000000000002e-06 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079830+0000", 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089749040000000006 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079833+0000", 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102327 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071891+0000", 2024-09-16T11:14:22.933 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45973904799999998, 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46840590300000001, 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102327 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071891+0000", 2024-09-16T11:14:22.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071891+0000", 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071892+0000", 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.046e-06 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071900+0000", 2024-09-16T11:14:22.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.191e-06 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071910+0000", 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9820000000000003e-06 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071915+0000", 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7489999999999998e-06 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080960+0000", 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009045371 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080964+0000", 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0940000000000001e-06 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102508 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072666+0000", 2024-09-16T11:14:22.937 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45896402800000002, 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46772451399999998, 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102508 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072666+0000", 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072666+0000", 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072666+0000", 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4200000000000002e-07 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072668+0000", 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.057e-06 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072674+0000", 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9290000000000003e-06 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072678+0000", 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5080000000000001e-06 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081928+0000", 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092501470000000002 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081931+0000", 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.461e-06 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45749117700000003, 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46634568500000001, 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.941 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:22.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:22.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:22.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101561 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066782+0000", 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46484777199999999, 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47379310400000002, 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101561 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066782+0000", 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066782+0000", 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066782+0000", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9000000000000002e-07 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066785+0000", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3250000000000002e-06 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066792+0000", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4170000000000003e-06 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066800+0000", 2024-09-16T11:14:22.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0339999999999997e-06 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076177+0000", 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093764690000000001 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076181+0000", 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8679999999999998e-06 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101637 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067666+0000", 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46396409900000002, 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47300370000000003, 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101637 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067666+0000", 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067666+0000", 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067666+0000", 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6799999999999999e-07 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067669+0000", 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0669999999999999e-06 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067679+0000", 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0886999999999999e-05 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067684+0000", 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.211e-06 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077003+0000", 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093193749999999995 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.369e-06 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101785 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068478+0000", 2024-09-16T11:14:22.952 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46315211099999998, 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47228462900000001, 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101785 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068478+0000", 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068478+0000", 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068478+0000", 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1899999999999998e-07 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068480+0000", 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1380000000000001e-06 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068486+0000", 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.784e-06 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068490+0000", 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.698e-06 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094871020000000007 2024-09-16T11:14:22.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077981+0000", 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0199999999999996e-06 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101833 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069293+0000", 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46233639399999998, 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47156421399999998, 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.956 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101833 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069293+0000", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069293+0000", 2024-09-16T11:14:22.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069294+0000", 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069296+0000", 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2010000000000002e-06 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069302+0000", 2024-09-16T11:14:22.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7169999999999996e-06 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069306+0000", 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.754e-06 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078915+0000", 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096095920000000001 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078919+0000", 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6160000000000002e-06 2024-09-16T11:14:22.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102074 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070868+0000", 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46076171300000002, 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.470079894, 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.960 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102074 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070868+0000", 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070868+0000", 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070869+0000", 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9399999999999995e-07 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070870+0000", 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070873+0000", 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2399999999999999e-06 2024-09-16T11:14:22.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070875+0000", 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079860+0000", 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0089849409999999998 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079866+0000", 2024-09-16T11:14:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6670000000000001e-06 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102329 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071920+0000", 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45970941599999998, 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46912516900000001, 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102329 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071920+0000", 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071920+0000", 2024-09-16T11:14:22.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071921+0000", 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6499999999999999e-07 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071923+0000", 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071929+0000", 2024-09-16T11:14:22.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1210000000000003e-06 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071934+0000", 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.566e-06 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080989+0000", 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090554539999999992 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080993+0000", 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5760000000000002e-06 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102509 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072684+0000", 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.458945294, 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46845505199999998, 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102509 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072684+0000", 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072684+0000", 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072685+0000", 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.65e-07 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072687+0000", 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.385e-06 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072693+0000", 2024-09-16T11:14:22.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8320000000000002e-06 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072697+0000", 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8480000000000003e-06 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081955+0000", 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092583400000000003 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081959+0000", 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5619999999999999e-06 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102616 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073371+0000", 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45825871099999999, 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46785927300000002, 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.972 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102616 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073371+0000", 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073371+0000", 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073372+0000", 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1399999999999997e-07 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073373+0000", 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4169999999999999e-06 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073379+0000", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7679999999999997e-06 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073383+0000", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.561e-06 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082624+0000", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092408439999999998 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082628+0000", 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.523e-06 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45747233799999998, 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46716673199999997, 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101563 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066805+0000", 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46482502799999997, 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47461050999999999, 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101563 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066805+0000", 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066805+0000", 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066805+0000", 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066820+0000", 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5034999999999999e-05 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066827+0000", 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3300000000000004e-06 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066833+0000", 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:22.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076206+0000", 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093725830000000003 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076210+0000", 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2549999999999999e-06 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101638 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067688+0000", 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46394175500000001, 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47383376900000002, 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101638 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067688+0000", 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067688+0000", 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067688+0000", 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3000000000000001e-07 2024-09-16T11:14:22.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067691+0000", 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3499999999999999e-06 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067701+0000", 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0475000000000001e-05 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067706+0000", 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3229999999999999e-06 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077036+0000", 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093302100000000002 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4170000000000001e-06 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101786 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068494+0000", 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46313595000000002, 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47311930800000002, 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101786 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068494+0000", 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068494+0000", 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068494+0000", 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5499999999999999e-07 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068496+0000", 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068507+0000", 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0312000000000001e-05 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068511+0000", 2024-09-16T11:14:22.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2409999999999997e-06 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077994+0000", 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094834069999999993 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077999+0000", 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5390000000000001e-06 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101834 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069313+0000", 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46231657999999998, 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472394443, 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101834 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069313+0000", 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069313+0000", 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069314+0000", 2024-09-16T11:14:22.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6300000000000001e-07 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069316+0000", 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.046e-06 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069322+0000", 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0009999999999998e-06 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069327+0000", 2024-09-16T11:14:22.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7370000000000003e-06 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078945+0000", 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096179270000000001 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078949+0000", 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4089999999999999e-06 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102075 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070888+0000", 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46074201599999998, 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47091401399999999, 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102075 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:22.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070888+0000", 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070888+0000", 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070888+0000", 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:22.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070890+0000", 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.353e-06 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070893+0000", 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1549999999999999e-06 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070895+0000", 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079908+0000", 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090130809999999992 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079912+0000", 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4709999999999999e-06 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:22.999 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102331 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071956+0000", 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45967391000000002, 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.469937259, 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102331 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071956+0000", 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071956+0000", 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071957+0000", 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4099999999999998e-07 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071959+0000", 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7939999999999998e-06 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071966+0000", 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0129999999999996e-06 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071971+0000", 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3429999999999998e-06 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081021+0000", 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090502039999999992 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081024+0000", 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4209999999999999e-06 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102510 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072711+0000", 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45891884100000002, 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46928298200000002, 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102510 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072711+0000", 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072711+0000", 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072712+0000", 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4600000000000004e-07 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072714+0000", 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6249999999999999e-06 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072722+0000", 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3520000000000001e-06 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072726+0000", 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2400000000000001e-06 2024-09-16T11:14:23.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081986+0000", 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092602410000000007 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081989+0000", 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45728038900000001, 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46767652700000001, 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:23.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.011 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101565 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066868+0000", 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46476176699999999, 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47519136299999998, 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101565 2024-09-16T11:14:23.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066868+0000", 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066868+0000", 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066869+0000", 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6199999999999997e-07 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066871+0000", 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5550000000000001e-06 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066878+0000", 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3260000000000001e-06 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066883+0000", 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.942e-06 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076240+0000", 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093565769999999996 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076244+0000", 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9389999999999999e-06 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.015 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101640 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067710+0000", 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46391989300000003, 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47438140099999998, 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.016 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101640 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067710+0000", 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067710+0000", 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067710+0000", 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067713+0000", 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.362e-06 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067718+0000", 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2259999999999996e-06 2024-09-16T11:14:23.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067722+0000", 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8959999999999996e-06 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093410339999999998 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077066+0000", 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.715e-06 2024-09-16T11:14:23.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101787 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068515+0000", 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46311465299999999, 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47360971299999999, 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.020 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101787 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068515+0000", 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068515+0000", 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068515+0000", 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.08e-07 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068518+0000", 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4030000000000001e-06 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068526+0000", 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5809999999999997e-06 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068529+0000", 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6620000000000001e-06 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078022+0000", 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094932839999999994 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078029+0000", 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7900000000000002e-06 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.023 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101835 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069331+0000", 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46229833300000001, 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47293627199999999, 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.024 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101835 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069331+0000", 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069331+0000", 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069332+0000", 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.71e-07 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069345+0000", 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2785999999999999e-05 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069351+0000", 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2029999999999997e-06 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069357+0000", 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6130000000000003e-06 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078973+0000", 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096163810000000002 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078977+0000", 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6229999999999999e-06 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.027 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102076 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070907+0000", 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46072255200000001, 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47139418399999999, 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102076 2024-09-16T11:14:23.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070907+0000", 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070907+0000", 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070908+0000", 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3099999999999998e-07 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070909+0000", 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.392e-06 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070914+0000", 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.459e-06 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070917+0000", 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5319999999999998e-06 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079939+0000", 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090220319999999993 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079945+0000", 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4859999999999997e-06 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.031 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102332 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071976+0000", 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45965352300000001, 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47035924499999998, 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102332 2024-09-16T11:14:23.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071976+0000", 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071976+0000", 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071977+0000", 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3099999999999997e-07 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071979+0000", 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9180000000000001e-06 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071984+0000", 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6169999999999998e-06 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071988+0000", 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9260000000000002e-06 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081051+0000", 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090626470000000001 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081055+0000", 2024-09-16T11:14:23.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6600000000000001e-06 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102511 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072730+0000", 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45889959400000002, 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.469637368, 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.036 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102511 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072730+0000", 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072730+0000", 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072730+0000", 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072733+0000", 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3530000000000002e-06 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072738+0000", 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6550000000000001e-06 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072742+0000", 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8639999999999998e-06 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082013+0000", 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092707169999999995 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082017+0000", 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7280000000000001e-06 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102618 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073409+0000", 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45822065499999998, 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46899318000000001, 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.040 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102618 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073409+0000", 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073409+0000", 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073410+0000", 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.44e-07 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073412+0000", 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.03e-06 2024-09-16T11:14:23.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073419+0000", 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2899999999999997e-06 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073423+0000", 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9040000000000002e-06 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082652+0000", 2024-09-16T11:14:23.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092291449999999994 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082655+0000", 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.32e-06 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.457261109, 2024-09-16T11:14:23.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46806546399999999, 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:23.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101567 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.048 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066894+0000", 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46473578999999998, 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47557389300000003, 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101567 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066894+0000", 2024-09-16T11:14:23.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066894+0000", 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066895+0000", 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5039999999999999e-06 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066898+0000", 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.367e-06 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066904+0000", 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1809999999999998e-06 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066908+0000", 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2359999999999998e-06 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076277+0000", 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093690370000000002 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076283+0000", 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3249999999999998e-06 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101642 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067727+0000", 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46390308299999999, 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47477424200000001, 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101642 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067727+0000", 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067727+0000", 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067727+0000", 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4300000000000003e-07 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067729+0000", 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0899999999999999e-06 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067738+0000", 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067743+0000", 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6310000000000002e-06 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077094+0000", 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093511800000000006 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077097+0000", 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3890000000000001e-06 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101788 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068532+0000", 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46309791099999997, 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47400142200000001, 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101788 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068532+0000", 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068532+0000", 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068532+0000", 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7399999999999999e-07 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068534+0000", 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.486e-06 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068538+0000", 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4499999999999997e-06 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068542+0000", 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8130000000000002e-06 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078055+0000", 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095134399999999997 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078059+0000", 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4249999999999998e-06 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101836 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069361+0000", 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46226826900000001, 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47320538499999998, 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101836 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069361+0000", 2024-09-16T11:14:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069361+0000", 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069362+0000", 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999998e-07 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069364+0000", 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069371+0000", 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.075e-06 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069375+0000", 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7699999999999999e-06 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079001+0000", 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096260570000000004 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079006+0000", 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2959999999999998e-06 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102077 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070923+0000", 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46070682200000002, 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47167589999999998, 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102077 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070923+0000", 2024-09-16T11:14:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070923+0000", 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070923+0000", 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6499999999999999e-07 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070925+0000", 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.502e-06 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070929+0000", 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4309999999999996e-06 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070932+0000", 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3700000000000002e-06 2024-09-16T11:14:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079971+0000", 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090391740000000005 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079974+0000", 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3579999999999999e-06 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102333 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071993+0000", 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45963706199999999, 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.470639535, 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102333 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.069 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071993+0000", 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071993+0000", 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071993+0000", 2024-09-16T11:14:23.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071995+0000", 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3539999999999998e-06 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072002+0000", 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2550000000000003e-06 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072006+0000", 2024-09-16T11:14:23.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5220000000000001e-06 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081079+0000", 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090730169999999992 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081083+0000", 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4300000000000002e-06 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.072 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102512 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072746+0000", 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45888341300000002, 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46991789699999997, 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102512 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.073 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072746+0000", 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072746+0000", 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072747+0000", 2024-09-16T11:14:23.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9400000000000001e-07 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072753+0000", 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5339999999999996e-06 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072759+0000", 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5369999999999997e-06 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072763+0000", 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3440000000000003e-06 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082047+0000", 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092840079999999998 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082051+0000", 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3780000000000001e-06 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.076 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102698 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077778+0000", 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45385204200000001, 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46492006000000002, 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102698 2024-09-16T11:14:23.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077778+0000", 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077778+0000", 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077778+0000", 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077780+0000", 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.663e-06 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077784+0000", 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5809999999999998e-06 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077787+0000", 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6249999999999999e-06 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083424+0000", 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056374249999999997 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083428+0000", 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3749999999999999e-06 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101569 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066919+0000", 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464710554, 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47581633099999998, 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101569 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066919+0000", 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066919+0000", 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066920+0000", 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9100000000000003e-07 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066922+0000", 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.328e-06 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066928+0000", 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9349999999999999e-06 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066932+0000", 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9770000000000002e-06 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076299+0000", 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093672900000000003 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076304+0000", 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5789999999999997e-06 2024-09-16T11:14:23.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101644 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067747+0000", 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46388248399999998, 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47502037499999999, 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.085 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101644 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067747+0000", 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067747+0000", 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067748+0000", 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9799999999999999e-07 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067750+0000", 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0310000000000001e-06 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067756+0000", 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0480000000000004e-06 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067759+0000", 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7409999999999998e-06 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077122+0000", 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093623449999999993 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077125+0000", 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:14:23.088 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101789 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068545+0000", 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463084357, 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47425531399999998, 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101789 2024-09-16T11:14:23.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068545+0000", 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068545+0000", 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068546+0000", 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5600000000000005e-07 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068548+0000", 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9740000000000001e-06 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068554+0000", 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8599999999999998e-06 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068558+0000", 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7929999999999999e-06 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078082+0000", 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095246459999999995 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078086+0000", 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.092 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101837 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069379+0000", 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.462250577, 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47345329000000003, 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101837 2024-09-16T11:14:23.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069379+0000", 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069379+0000", 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069380+0000", 2024-09-16T11:14:23.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2900000000000003e-07 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069382+0000", 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0980000000000001e-06 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069388+0000", 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8889999999999999e-06 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069392+0000", 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9369999999999997e-06 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079036+0000", 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096439009999999999 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079039+0000", 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6289999999999999e-06 2024-09-16T11:14:23.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102078 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070939+0000", 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46069072799999999, 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47192656100000002, 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.097 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102078 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070939+0000", 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070939+0000", 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070939+0000", 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6600000000000003e-07 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070941+0000", 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4890000000000001e-06 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070944+0000", 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0730000000000001e-06 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070946+0000", 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2060000000000001e-06 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080000+0000", 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090542469999999996 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080005+0000", 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7369999999999997e-06 2024-09-16T11:14:23.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102334 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072026+0000", 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45960375799999997, 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.470871129, 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.101 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102334 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072026+0000", 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072026+0000", 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072027+0000", 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.61e-07 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072029+0000", 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.48e-06 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072036+0000", 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0450000000000003e-06 2024-09-16T11:14:23.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072040+0000", 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0849999999999999e-06 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081109+0000", 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00906912 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081113+0000", 2024-09-16T11:14:23.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.67e-06 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102513 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072768+0000", 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45886219, 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47016295899999999, 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102513 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072768+0000", 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072768+0000", 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072768+0000", 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4700000000000002e-07 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072770+0000", 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5660000000000001e-06 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072777+0000", 2024-09-16T11:14:23.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0789999999999997e-06 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072780+0000", 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6459999999999999e-06 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082077+0000", 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092966070000000001 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082080+0000", 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6880000000000001e-06 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102700 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077800+0000", 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.453829343, 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46516317699999998, 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102700 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077800+0000", 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077800+0000", 2024-09-16T11:14:23.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077801+0000", 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1800000000000002e-07 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077802+0000", 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.657e-06 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077806+0000", 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6260000000000001e-06 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077808+0000", 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3120000000000001e-06 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083456+0000", 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056481370000000001 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083460+0000", 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5219999999999999e-06 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101571 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066951+0000", 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464679073, 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47604490599999999, 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.113 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101571 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066951+0000", 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066951+0000", 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066951+0000", 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5900000000000002e-07 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066954+0000", 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4679999999999999e-06 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066960+0000", 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1550000000000004e-06 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066970+0000", 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9189999999999994e-06 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076326+0000", 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093564670000000003 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076330+0000", 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5999999999999998e-06 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101646 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067764+0000", 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46386613700000001, 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47526555599999998, 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.117 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101646 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067764+0000", 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067764+0000", 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067764+0000", 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067776+0000", 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.172e-05 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067783+0000", 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0049999999999998e-06 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067787+0000", 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5379999999999996e-06 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077154+0000", 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093665150000000006 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077157+0000", 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4319999999999999e-06 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101790 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068562+0000", 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463068062, 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47449949000000002, 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101790 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068562+0000", 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068562+0000", 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068562+0000", 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8500000000000002e-07 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068564+0000", 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.86e-06 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068573+0000", 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3070000000000008e-06 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068577+0000", 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.67e-06 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078113+0000", 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095363549999999998 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078119+0000", 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0270000000000001e-06 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101838 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069396+0000", 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46223384899999997, 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47369863299999998, 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.125 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101838 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069396+0000", 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069396+0000", 2024-09-16T11:14:23.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069396+0000", 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069398+0000", 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9e-06 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069409+0000", 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0842999999999999e-05 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069413+0000", 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2679999999999996e-06 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079064+0000", 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096503619999999991 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079069+0000", 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3279999999999996e-06 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102079 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070966+0000", 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46066366399999997, 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472160364, 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.129 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102079 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070966+0000", 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070966+0000", 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070967+0000", 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.89e-07 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070968+0000", 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.423e-06 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070971+0000", 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.247e-06 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070973+0000", 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2749999999999998e-06 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080032+0000", 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090584380000000003 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080035+0000", 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4419999999999998e-06 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102335 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072045+0000", 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.459585138, 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47111661199999999, 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.133 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102335 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072045+0000", 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072045+0000", 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072056+0000", 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1793e-05 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072059+0000", 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.757e-06 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072066+0000", 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7000000000000002e-06 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072071+0000", 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8799999999999999e-06 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081137+0000", 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090662829999999996 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081140+0000", 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4020000000000002e-06 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102514 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072784+0000", 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45884550299999999, 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47041054900000001, 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.137 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102514 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072784+0000", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072784+0000", 2024-09-16T11:14:23.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072785+0000", 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5999999999999999e-07 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072786+0000", 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7749999999999999e-06 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072795+0000", 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0459999999999994e-06 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072799+0000", 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7419999999999999e-06 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082106+0000", 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093066910000000006 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082112+0000", 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102702 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077810+0000", 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45381927500000002, 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46541656199999998, 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102702 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077810+0000", 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077810+0000", 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077811+0000", 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1300000000000001e-07 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077812+0000", 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2669999999999999e-06 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077816+0000", 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.608e-06 2024-09-16T11:14:23.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077818+0000", 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6560000000000001e-06 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083483+0000", 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056647240000000003 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083486+0000", 2024-09-16T11:14:23.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3629999999999998e-06 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101573 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.066981+0000", 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46464852800000001, 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47627915399999998, 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101573 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066981+0000", 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066981+0000", 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066982+0000", 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4600000000000005e-07 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066984+0000", 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066991+0000", 2024-09-16T11:14:23.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.596e-06 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.066994+0000", 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.738e-06 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076350+0000", 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093558649999999997 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076354+0000", 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5049999999999999e-06 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101647 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067792+0000", 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46383759200000002, 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47550009199999999, 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.149 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101647 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067792+0000", 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067792+0000", 2024-09-16T11:14:23.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067793+0000", 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499999999999998e-07 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067795+0000", 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2529999999999999e-06 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067803+0000", 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067810+0000", 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1860000000000003e-06 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077181+0000", 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093709819999999999 2024-09-16T11:14:23.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077185+0000", 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101791 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068601+0000", 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46302860000000001, 2024-09-16T11:14:23.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47472447800000001, 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101791 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068601+0000", 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068601+0000", 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068601+0000", 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2099999999999998e-07 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068603+0000", 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.55e-06 2024-09-16T11:14:23.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068607+0000", 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8859999999999997e-06 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068610+0000", 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6390000000000001e-06 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078144+0000", 2024-09-16T11:14:23.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095347720000000004 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078148+0000", 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7859999999999998e-06 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101839 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069417+0000", 2024-09-16T11:14:23.157 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46221238599999998, 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47394002499999999, 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101839 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069417+0000", 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069417+0000", 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069418+0000", 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.96e-07 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069420+0000", 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-06 2024-09-16T11:14:23.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069425+0000", 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3959999999999999e-06 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069427+0000", 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2359999999999999e-06 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079097+0000", 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096701359999999993 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079101+0000", 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5609999999999999e-06 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102080 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070976+0000", 2024-09-16T11:14:23.161 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46065391, 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47241529500000001, 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102080 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070976+0000", 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070976+0000", 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070976+0000", 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8299999999999997e-07 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070978+0000", 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3790000000000001e-06 2024-09-16T11:14:23.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070981+0000", 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9280000000000002e-06 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070984+0000", 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080066+0000", 2024-09-16T11:14:23.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090816160000000007 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080069+0000", 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6839999999999998e-06 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102336 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072076+0000", 2024-09-16T11:14:23.165 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45955384799999999, 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47134889400000002, 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102336 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072076+0000", 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072076+0000", 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072078+0000", 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4080000000000001e-06 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072081+0000", 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4870000000000001e-06 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072097+0000", 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6121999999999999e-05 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072101+0000", 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4780000000000002e-06 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081167+0000", 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090660289999999998 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081171+0000", 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.169 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102515 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072803+0000", 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45882635799999999, 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47065374199999999, 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102515 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072803+0000", 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072803+0000", 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072804+0000", 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072806+0000", 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2840000000000001e-06 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072812+0000", 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5969999999999999e-06 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072815+0000", 2024-09-16T11:14:23.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6289999999999999e-06 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082141+0000", 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093260340000000004 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082145+0000", 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6660000000000001e-06 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102704 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077820+0000", 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45380930800000002, 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.465670264, 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102704 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077820+0000", 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077820+0000", 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077821+0000", 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8799999999999998e-07 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077822+0000", 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.53e-06 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077825+0000", 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077827+0000", 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0549999999999998e-06 2024-09-16T11:14:23.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083513+0000", 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056856249999999997 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083517+0000", 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2189999999999998e-06 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.177 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101575 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067023+0000", 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46460682800000003, 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47650048499999997, 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101575 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067023+0000", 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067023+0000", 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067024+0000", 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7400000000000003e-07 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067026+0000", 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2630000000000002e-06 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067032+0000", 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8629999999999996e-06 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067036+0000", 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1640000000000003e-06 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076371+0000", 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093349769999999995 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076374+0000", 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.495e-06 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.181 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101649 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067815+0000", 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46381480200000003, 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47574154600000002, 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101649 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067815+0000", 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067815+0000", 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067815+0000", 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067818+0000", 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2500000000000001e-06 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067824+0000", 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9660000000000001e-06 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067827+0000", 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5269999999999999e-06 2024-09-16T11:14:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077210+0000", 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093827979999999995 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077213+0000", 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4249999999999998e-06 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101792 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068632+0000", 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46299773100000002, 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47495649200000001, 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101792 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068632+0000", 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068632+0000", 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068633+0000", 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5400000000000001e-07 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068635+0000", 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.967e-06 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068641+0000", 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0159999999999997e-06 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068645+0000", 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6310000000000002e-06 2024-09-16T11:14:23.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078171+0000", 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095252759999999992 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078174+0000", 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3809999999999999e-06 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101840 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069444+0000", 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46218591599999997, 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47417820999999999, 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101840 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069444+0000", 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069444+0000", 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069444+0000", 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.91e-07 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069446+0000", 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5060000000000001e-06 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069451+0000", 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.383e-06 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069453+0000", 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5000000000000002e-06 2024-09-16T11:14:23.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079125+0000", 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096712659999999995 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079128+0000", 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102081 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.070986+0000", 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46064340500000001, 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47266902900000002, 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102081 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070986+0000", 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070986+0000", 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070987+0000", 2024-09-16T11:14:23.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5899999999999998e-07 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070991+0000", 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000001e-06 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070994+0000", 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1439999999999999e-06 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.070998+0000", 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3569999999999998e-06 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080094+0000", 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009096102 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080097+0000", 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:14:23.197 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102337 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072107+0000", 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45952305700000001, 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47158071000000001, 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.198 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102337 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072107+0000", 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072107+0000", 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072108+0000", 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1999999999999998e-07 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072110+0000", 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3199999999999998e-06 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072117+0000", 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5510000000000002e-06 2024-09-16T11:14:23.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072122+0000", 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1989999999999999e-06 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081196+0000", 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090740669999999999 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081199+0000", 2024-09-16T11:14:23.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102516 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072819+0000", 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45881037899999999, 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47090122400000001, 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102516 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072819+0000", 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072819+0000", 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072820+0000", 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072822+0000", 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0880000000000002e-06 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072828+0000", 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8189999999999997e-06 2024-09-16T11:14:23.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072831+0000", 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7400000000000002e-06 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082170+0000", 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093381660000000002 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082175+0000", 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5720000000000002e-06 2024-09-16T11:14:23.205 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45379084400000003, 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.465913517, 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.206 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:23.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:23.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101597 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067040+0000", 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46458978099999998, 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47674602300000002, 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.210 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101597 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067040+0000", 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067040+0000", 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067040+0000", 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067042+0000", 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1859999999999999e-06 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067054+0000", 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1289e-05 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067058+0000", 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9960000000000004e-06 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076398+0000", 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093398930000000002 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076401+0000", 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6669999999999998e-06 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.213 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101713 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067831+0000", 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46379830999999999, 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47598637599999999, 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101713 2024-09-16T11:14:23.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067831+0000", 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067831+0000", 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067832+0000", 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067834+0000", 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067842+0000", 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7710000000000006e-06 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067847+0000", 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.429e-06 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077238+0000", 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009390776 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077242+0000", 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6219999999999998e-06 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.217 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101793 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068649+0000", 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46298056100000001, 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47521075099999999, 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101793 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.218 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068649+0000", 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068649+0000", 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068650+0000", 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9400000000000005e-07 2024-09-16T11:14:23.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068652+0000", 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.767e-06 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068657+0000", 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6930000000000004e-06 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068660+0000", 2024-09-16T11:14:23.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6180000000000002e-06 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078196+0000", 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095364839999999996 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078200+0000", 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5769999999999998e-06 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101841 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069456+0000", 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46217373099999998, 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47443882300000001, 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101841 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069456+0000", 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069456+0000", 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069456+0000", 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:14:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069458+0000", 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.331e-06 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069461+0000", 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1650000000000002e-06 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069463+0000", 2024-09-16T11:14:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2689999999999998e-06 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079155+0000", 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0096921460000000004 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079159+0000", 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7129999999999999e-06 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102082 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071000+0000", 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46062998799999999, 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472926925, 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102082 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071000+0000", 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071000+0000", 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071000+0000", 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0500000000000004e-07 2024-09-16T11:14:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071002+0000", 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.305e-06 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071004+0000", 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9160000000000001e-06 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071007+0000", 2024-09-16T11:14:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1780000000000002e-06 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080124+0000", 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091177759999999993 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080128+0000", 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7440000000000001e-06 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102338 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072126+0000", 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.459503832, 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47183429199999999, 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102338 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072126+0000", 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072126+0000", 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072127+0000", 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5899999999999996e-07 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072129+0000", 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0940000000000002e-06 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072135+0000", 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0650000000000004e-06 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072139+0000", 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9230000000000004e-06 2024-09-16T11:14:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081221+0000", 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090826580000000004 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081227+0000", 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9739999999999999e-06 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.453769541, 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46613162499999999, 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101599 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067070+0000", 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464560113, 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47695694100000002, 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101599 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067070+0000", 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067070+0000", 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067070+0000", 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067071+0000", 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.471e-06 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067075+0000", 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6669999999999998e-06 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067077+0000", 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.446e-06 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076424+0000", 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093467670000000006 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076428+0000", 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5480000000000002e-06 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101715 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067851+0000", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463778735, 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47620771299999998, 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101715 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067851+0000", 2024-09-16T11:14:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067851+0000", 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067852+0000", 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8500000000000001e-07 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067854+0000", 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3120000000000001e-06 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067860+0000", 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1500000000000004e-06 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067864+0000", 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9260000000000002e-06 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077268+0000", 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094038539999999997 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077271+0000", 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101794 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068691+0000", 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46293887700000003, 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47540185699999998, 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101794 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068691+0000", 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068691+0000", 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068691+0000", 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7400000000000003e-07 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068694+0000", 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.136e-06 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068699+0000", 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2070000000000003e-06 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068704+0000", 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0579999999999997e-06 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078224+0000", 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095206179999999998 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078228+0000", 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3739999999999998e-06 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101842 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069465+0000", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46216432800000001, 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47466077699999998, 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101842 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069465+0000", 2024-09-16T11:14:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069465+0000", 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069466+0000", 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6100000000000002e-07 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069468+0000", 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9889999999999999e-06 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069474+0000", 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7810000000000002e-06 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069477+0000", 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8639999999999998e-06 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079183+0000", 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097058690000000006 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079187+0000", 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3500000000000001e-06 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102083 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071017+0000", 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46061296099999999, 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47314129100000002, 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102083 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071017+0000", 2024-09-16T11:14:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071017+0000", 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071017+0000", 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4499999999999998e-07 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071018+0000", 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3999999999999999e-06 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071022+0000", 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1810000000000001e-06 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071024+0000", 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080154+0000", 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091305039999999994 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080158+0000", 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102339 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072152+0000", 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45947809899999997, 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47203980000000001, 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102339 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072152+0000", 2024-09-16T11:14:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072152+0000", 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072152+0000", 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8899999999999999e-07 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072154+0000", 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2390000000000001e-06 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072161+0000", 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6379999999999998e-06 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072168+0000", 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9999999999999999e-06 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081253+0000", 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090849710000000007 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081256+0000", 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4350000000000001e-06 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45375947, 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46635314700000002, 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101600 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067081+0000", 2024-09-16T11:14:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464548884, 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.477176025, 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101600 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067081+0000", 2024-09-16T11:14:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067081+0000", 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067081+0000", 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067084+0000", 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0430000000000002e-06 2024-09-16T11:14:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067089+0000", 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5990000000000001e-06 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067093+0000", 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2209999999999999e-06 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076448+0000", 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093550600000000001 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076452+0000", 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7019999999999999e-06 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101718 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067868+0000", 2024-09-16T11:14:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46376189299999998, 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47642049400000003, 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101718 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067868+0000", 2024-09-16T11:14:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067868+0000", 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067868+0000", 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067870+0000", 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0379999999999998e-06 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067882+0000", 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1714e-05 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067886+0000", 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9740000000000004e-06 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077297+0000", 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094105720000000007 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077300+0000", 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101795 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068730+0000", 2024-09-16T11:14:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46289935100000001, 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47559119500000002, 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101795 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068730+0000", 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068730+0000", 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068731+0000", 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8500000000000001e-07 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068737+0000", 2024-09-16T11:14:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.964e-06 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068749+0000", 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1559999999999999e-05 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068753+0000", 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2520000000000001e-06 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078253+0000", 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094996799999999999 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078256+0000", 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6500000000000002e-06 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101843 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069499+0000", 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46213073700000001, 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47485630600000001, 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101843 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069499+0000", 2024-09-16T11:14:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069499+0000", 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069500+0000", 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3e-07 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069502+0000", 2024-09-16T11:14:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.509e-06 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069508+0000", 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1750000000000002e-06 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069511+0000", 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5459999999999998e-06 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079213+0000", 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097020219999999994 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079217+0000", 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6969999999999999e-06 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102084 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071026+0000", 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46060376200000003, 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47336128599999999, 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102084 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071026+0000", 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071026+0000", 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071026+0000", 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.48e-07 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071028+0000", 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071030+0000", 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8679999999999999e-06 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071033+0000", 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1569999999999998e-06 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080184+0000", 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091514509999999997 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080188+0000", 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6720000000000002e-06 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102341 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072174+0000", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45945620199999998, 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47224704000000001, 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102341 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072174+0000", 2024-09-16T11:14:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072174+0000", 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072174+0000", 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8299999999999997e-07 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072176+0000", 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.984e-06 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072182+0000", 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0460000000000002e-06 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072186+0000", 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4150000000000001e-06 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081283+0000", 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090961229999999994 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081288+0000", 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6559999999999997e-06 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102634 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073598+0000", 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45803137700000002, 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47085394400000002, 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102634 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073598+0000", 2024-09-16T11:14:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073598+0000", 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073599+0000", 2024-09-16T11:14:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7199999999999999e-07 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073601+0000", 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.88e-06 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073607+0000", 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7100000000000004e-06 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073610+0000", 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7890000000000001e-06 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082683+0000", 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090726669999999995 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082687+0000", 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5619999999999999e-06 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45373116699999999, 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.466589484, 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101602 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067105+0000", 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46452520899999999, 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.477415319, 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101602 2024-09-16T11:14:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067105+0000", 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067105+0000", 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067105+0000", 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0999999999999999e-07 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067107+0000", 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2089999999999999e-06 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067113+0000", 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1349999999999997e-06 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067117+0000", 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8850000000000001e-06 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076471+0000", 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093535059999999993 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076474+0000", 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5899999999999999e-06 2024-09-16T11:14:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101720 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067913+0000", 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463716299, 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47663984100000001, 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101720 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067913+0000", 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067913+0000", 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067914+0000", 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0900000000000001e-07 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067916+0000", 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2989999999999999e-06 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067924+0000", 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067928+0000", 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0620000000000002e-06 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077323+0000", 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093949299999999993 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077327+0000", 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:14:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101796 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068757+0000", 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46287235199999999, 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.475828949, 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101796 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068757+0000", 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068757+0000", 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068758+0000", 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8599999999999999e-07 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068760+0000", 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068770+0000", 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0148e-05 2024-09-16T11:14:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068775+0000", 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6330000000000004e-06 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078283+0000", 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095087069999999999 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078287+0000", 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.416e-06 2024-09-16T11:14:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101844 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069514+0000", 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46211578800000003, 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47510407100000002, 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101844 2024-09-16T11:14:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069514+0000", 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069514+0000", 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069514+0000", 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.51e-07 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069516+0000", 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9939999999999999e-06 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069522+0000", 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9610000000000002e-06 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069526+0000", 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4529999999999998e-06 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079241+0000", 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097146569999999998 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079244+0000", 2024-09-16T11:14:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3950000000000001e-06 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102085 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071048+0000", 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46058126999999999, 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47360287000000001, 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102085 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071048+0000", 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071048+0000", 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071049+0000", 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8699999999999995e-07 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071050+0000", 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.415e-06 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071054+0000", 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2009999999999999e-06 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071056+0000", 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7199999999999998e-06 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080213+0000", 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009156539 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080219+0000", 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102344 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072191+0000", 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45943887999999999, 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47249229799999998, 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102344 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072191+0000", 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072191+0000", 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072191+0000", 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072206+0000", 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.519e-05 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072215+0000", 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3149999999999992e-06 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072220+0000", 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.843e-06 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081315+0000", 2024-09-16T11:14:23.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090953909999999995 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081318+0000", 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102595 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072908+0000", 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45872135600000002, 2024-09-16T11:14:23.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47180794100000001, 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102595 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072908+0000", 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072908+0000", 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072909+0000", 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072911+0000", 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072916+0000", 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8470000000000001e-06 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072920+0000", 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.811e-06 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082203+0000", 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092827399999999994 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082207+0000", 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6519999999999999e-06 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.326 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102635 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073615+0000", 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45801519299999999, 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47113391199999999, 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102635 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073615+0000", 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073615+0000", 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073615+0000", 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0100000000000005e-07 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073617+0000", 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9690000000000001e-06 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073623+0000", 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8270000000000003e-06 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073627+0000", 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.788e-06 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082712+0000", 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090853519999999997 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082715+0000", 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.348e-06 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45371723800000002, 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.466869058, 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:23.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101604 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067121+0000", 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46450878000000001, 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47769387299999999, 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101604 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067121+0000", 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067121+0000", 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067121+0000", 2024-09-16T11:14:23.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067124+0000", 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3779999999999999e-06 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067130+0000", 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9070000000000004e-06 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067134+0000", 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9949999999999999e-06 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076514+0000", 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093803800000000007 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076518+0000", 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.551e-06 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.338 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101722 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067933+0000", 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.463697199, 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47691858100000001, 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101722 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.339 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067933+0000", 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067933+0000", 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067933+0000", 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067935+0000", 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0849999999999999e-06 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067947+0000", 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2081e-05 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067952+0000", 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6319999999999999e-06 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077352+0000", 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094004789999999998 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077356+0000", 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101798 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068780+0000", 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46284992699999999, 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47610445299999998, 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101798 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.343 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068780+0000", 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068780+0000", 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068780+0000", 2024-09-16T11:14:23.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5499999999999998e-07 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068782+0000", 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.897e-06 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068789+0000", 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2210000000000002e-06 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068792+0000", 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8580000000000002e-06 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078315+0000", 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00952275 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078319+0000", 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639999999999999e-06 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101845 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069528+0000", 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46210147499999998, 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47538802099999999, 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101845 2024-09-16T11:14:23.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069528+0000", 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069528+0000", 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069529+0000", 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5800000000000001e-07 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069530+0000", 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3239999999999999e-06 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069533+0000", 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-06 2024-09-16T11:14:23.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069537+0000", 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8870000000000002e-06 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079273+0000", 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097361679999999999 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079277+0000", 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6349999999999999e-06 2024-09-16T11:14:23.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102086 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071078+0000", 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46055145800000002, 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47387122100000001, 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.351 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102086 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071078+0000", 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071078+0000", 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071079+0000", 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999998e-07 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071081+0000", 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.221e-06 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071089+0000", 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6660000000000008e-06 2024-09-16T11:14:23.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071093+0000", 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2520000000000001e-06 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080247+0000", 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091540259999999991 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080251+0000", 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6849999999999999e-06 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102346 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072224+0000", 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45940526199999998, 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47275631699999998, 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102346 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072224+0000", 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072224+0000", 2024-09-16T11:14:23.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072225+0000", 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5400000000000002e-07 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072227+0000", 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3700000000000002e-06 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072251+0000", 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3359000000000001e-05 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072255+0000", 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5530000000000003e-06 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081344+0000", 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090891489999999995 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081348+0000", 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102596 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072937+0000", 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45869301499999998, 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472077319, 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102596 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072937+0000", 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072937+0000", 2024-09-16T11:14:23.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072937+0000", 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1699999999999999e-07 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072940+0000", 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5639999999999999e-06 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072946+0000", 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.392e-06 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072950+0000", 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0090000000000001e-06 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082231+0000", 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009281035 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082235+0000", 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.687e-06 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102636 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073630+0000", 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45799936800000002, 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47141676100000002, 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102636 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073630+0000", 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073630+0000", 2024-09-16T11:14:23.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073631+0000", 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5799999999999999e-07 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073633+0000", 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9120000000000001e-06 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073639+0000", 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1580000000000002e-06 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073643+0000", 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7869999999999999e-06 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082744+0000", 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091006969999999996 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082747+0000", 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6030000000000001e-06 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.453708475, 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.467157869, 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:23.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:23.370 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101606 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067155+0000", 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46447444300000001, 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47795708799999997, 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101606 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.371 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067155+0000", 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067155+0000", 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067156+0000", 2024-09-16T11:14:23.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.75e-07 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067169+0000", 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2954e-05 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067175+0000", 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2299999999999996e-06 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067179+0000", 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076538+0000", 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093592209999999992 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076542+0000", 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6239999999999999e-06 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.374 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101724 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067964+0000", 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46366595999999999, 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47718023300000001, 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101724 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.375 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067964+0000", 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067964+0000", 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067964+0000", 2024-09-16T11:14:23.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067966+0000", 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2919999999999998e-06 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067973+0000", 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1979999999999998e-06 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067977+0000", 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2919999999999997e-06 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077380+0000", 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094027569999999994 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077383+0000", 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4850000000000001e-06 2024-09-16T11:14:23.378 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101800 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068798+0000", 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46283191499999998, 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47637956999999997, 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.379 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101800 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068798+0000", 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068798+0000", 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068798+0000", 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0900000000000002e-07 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068800+0000", 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068811+0000", 2024-09-16T11:14:23.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1141999999999999e-05 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068816+0000", 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.104e-06 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078345+0000", 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095295399999999995 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078348+0000", 2024-09-16T11:14:23.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3419999999999999e-06 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101846 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069559+0000", 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46207056499999999, 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47564981099999998, 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101846 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069559+0000", 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069559+0000", 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069560+0000", 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8299999999999997e-07 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069562+0000", 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2000000000000001e-06 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069567+0000", 2024-09-16T11:14:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.809e-06 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069570+0000", 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079301+0000", 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097308159999999998 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079304+0000", 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4640000000000002e-06 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102087 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071114+0000", 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.460515375, 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.474127768, 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.387 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102087 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071114+0000", 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071114+0000", 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071115+0000", 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2600000000000002e-07 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071118+0000", 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6139999999999999e-06 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071124+0000", 2024-09-16T11:14:23.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1789999999999996e-06 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071128+0000", 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9299999999999996e-06 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080275+0000", 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091471959999999998 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080281+0000", 2024-09-16T11:14:23.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8869999999999997e-06 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102348 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072260+0000", 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45936925899999997, 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.473014938, 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102348 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072260+0000", 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072260+0000", 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072261+0000", 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6600000000000002e-07 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072264+0000", 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.576e-06 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072278+0000", 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4374e-05 2024-09-16T11:14:23.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072283+0000", 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9910000000000002e-06 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081375+0000", 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090926080000000003 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081379+0000", 2024-09-16T11:14:23.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4359999999999998e-06 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102597 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072955+0000", 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45867491999999999, 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47235292099999998, 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.395 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102597 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072955+0000", 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072955+0000", 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072955+0000", 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-07 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072958+0000", 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.362e-06 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072963+0000", 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7860000000000002e-06 2024-09-16T11:14:23.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072967+0000", 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082259+0000", 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0092915180000000003 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082262+0000", 2024-09-16T11:14:23.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4690000000000002e-06 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102638 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073647+0000", 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45798273, 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47169376299999999, 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.399 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102638 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073647+0000", 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073647+0000", 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073647+0000", 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073649+0000", 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9410000000000001e-06 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073730+0000", 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0698999999999994e-05 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073736+0000", 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082772+0000", 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090357670000000001 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082775+0000", 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.36e-06 2024-09-16T11:14:23.402 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45368465600000002, 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.467427551, 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:14:23.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:23.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101607 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067184+0000", 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46444596799999999, 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47822184299999998, 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.407 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101607 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067184+0000", 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067184+0000", 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067184+0000", 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3600000000000004e-07 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067187+0000", 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.272e-06 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067193+0000", 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9519999999999999e-06 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067197+0000", 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.357e-06 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076562+0000", 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093648680000000002 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076565+0000", 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5370000000000002e-06 2024-09-16T11:14:23.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101726 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067981+0000", 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46364851899999998, 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47745653399999999, 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.411 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101726 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067981+0000", 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067981+0000", 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067982+0000", 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7700000000000004e-07 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067984+0000", 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067994+0000", 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0737000000000001e-05 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067998+0000", 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8940000000000003e-06 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077407+0000", 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009408942 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077411+0000", 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.631e-06 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.414 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101802 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068839+0000", 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46279069, 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47663165200000002, 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101802 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068839+0000", 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068839+0000", 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068839+0000", 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.79e-07 2024-09-16T11:14:23.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068841+0000", 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5600000000000001e-06 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068845+0000", 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.698e-06 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068847+0000", 2024-09-16T11:14:23.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.65e-06 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078373+0000", 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095254410000000008 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078376+0000", 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6160000000000002e-06 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101847 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069574+0000", 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.462056203, 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47593582499999998, 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101847 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069574+0000", 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069574+0000", 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069574+0000", 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069580+0000", 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4370000000000004e-06 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069593+0000", 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3033999999999999e-05 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069598+0000", 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0000000000000004e-06 2024-09-16T11:14:23.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079330+0000", 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097319769999999993 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079336+0000", 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8130000000000001e-06 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102088 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071152+0000", 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46047726999999999, 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47438879299999998, 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102088 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071152+0000", 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071152+0000", 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071153+0000", 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5499999999999997e-07 2024-09-16T11:14:23.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071156+0000", 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3829999999999999e-06 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071162+0000", 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.117e-06 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071166+0000", 2024-09-16T11:14:23.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9509999999999999e-06 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080309+0000", 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091431250000000002 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080312+0000", 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6330000000000002e-06 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102350 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072288+0000", 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45934219799999998, 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47328722000000001, 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102350 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072288+0000", 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072288+0000", 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072288+0000", 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2500000000000001e-07 2024-09-16T11:14:23.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072290+0000", 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2340000000000001e-06 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072297+0000", 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3770000000000001e-06 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072301+0000", 2024-09-16T11:14:23.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6770000000000003e-06 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081404+0000", 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091029830000000003 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081408+0000", 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9249999999999997e-06 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102598 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072971+0000", 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45865868399999998, 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47263548399999999, 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102598 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072971+0000", 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072971+0000", 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072971+0000", 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9000000000000002e-07 2024-09-16T11:14:23.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072974+0000", 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072979+0000", 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7169999999999996e-06 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072984+0000", 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6879999999999998e-06 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082287+0000", 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093032070000000008 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082291+0000", 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5360000000000001e-06 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.434 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102640 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073742+0000", 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.457887608, 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47189962699999999, 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102640 2024-09-16T11:14:23.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073742+0000", 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073742+0000", 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073743+0000", 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6499999999999998e-07 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073746+0000", 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2380000000000002e-06 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073754+0000", 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5809999999999999e-06 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073758+0000", 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0289999999999999e-06 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082803+0000", 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090450029999999994 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082806+0000", 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5939999999999998e-06 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45365353400000002, 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46769731599999997, 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.439 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:14:23.442 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101608 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067216+0000", 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.464413992, 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47849113199999999, 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.443 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101608 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067216+0000", 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067216+0000", 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067216+0000", 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5799999999999999e-07 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067219+0000", 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067224+0000", 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7910000000000001e-06 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067229+0000", 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4900000000000002e-06 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076585+0000", 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093564070000000006 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076589+0000", 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2940000000000001e-06 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.446 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101731 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068003+0000", 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46362705300000001, 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47773770799999998, 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101731 2024-09-16T11:14:23.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068003+0000", 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068003+0000", 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068003+0000", 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8599999999999999e-07 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068005+0000", 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1339999999999998e-06 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068015+0000", 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0224000000000001e-05 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068022+0000", 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2190000000000001e-06 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077435+0000", 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094135490000000002 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077439+0000", 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101804 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068869+0000", 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46276118999999999, 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47690377299999998, 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101804 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.451 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068869+0000", 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068869+0000", 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068869+0000", 2024-09-16T11:14:23.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7700000000000004e-07 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068871+0000", 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1969999999999999e-06 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068877+0000", 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8930000000000002e-06 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068882+0000", 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2230000000000001e-06 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078403+0000", 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095210420000000004 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078406+0000", 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3670000000000001e-06 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.454 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101849 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069619+0000", 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46201115100000001, 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47618758, 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101849 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.455 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069619+0000", 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069619+0000", 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069619+0000", 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9999999999999997e-07 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069622+0000", 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.283e-06 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069628+0000", 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9850000000000003e-06 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069632+0000", 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1239999999999998e-06 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079362+0000", 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097299699999999992 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079365+0000", 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.41e-06 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.458 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102089 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071178+0000", 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.460451477, 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47465972699999998, 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102089 2024-09-16T11:14:23.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071178+0000", 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071178+0000", 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071179+0000", 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3499999999999996e-07 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071181+0000", 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5780000000000001e-06 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071188+0000", 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4570000000000002e-06 2024-09-16T11:14:23.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071193+0000", 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1819999999999997e-06 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080337+0000", 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091439020000000006 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080340+0000", 2024-09-16T11:14:23.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4960000000000001e-06 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102359 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072305+0000", 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45932429499999999, 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47356572699999999, 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102359 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072305+0000", 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072305+0000", 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072306+0000", 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072308+0000", 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.827e-06 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072314+0000", 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.054e-06 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072318+0000", 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9840000000000003e-06 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081434+0000", 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091167450000000008 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081438+0000", 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102599 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072988+0000", 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45864185200000002, 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47291498399999998, 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102599 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072988+0000", 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072988+0000", 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072988+0000", 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073001+0000", 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2653000000000001e-05 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073009+0000", 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5979999999999999e-06 2024-09-16T11:14:23.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073012+0000", 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8720000000000004e-06 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082317+0000", 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093046269999999993 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082320+0000", 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3620000000000002e-06 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102642 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073762+0000", 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45786802500000001, 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.472174223, 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102642 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073762+0000", 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073762+0000", 2024-09-16T11:14:23.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073762+0000", 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5900000000000003e-07 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073764+0000", 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9570000000000001e-06 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073770+0000", 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4990000000000002e-06 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073773+0000", 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.766e-06 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082830+0000", 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090567430000000008 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082833+0000", 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4539999999999999e-06 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4.d885ae0d (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45360579499999998, 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.46815810299999999, 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:14:23.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:14:23.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101610 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067235+0000", 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46439462399999998, 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47898086000000001, 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.479 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101610 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067235+0000", 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067235+0000", 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067236+0000", 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-07 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067238+0000", 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0209999999999998e-06 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067244+0000", 2024-09-16T11:14:23.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9599999999999997e-06 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067247+0000", 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076609+0000", 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093623319999999993 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076613+0000", 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4829999999999999e-06 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101733 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068052+0000", 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46357808900000003, 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47819979299999998, 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101733 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068052+0000", 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068052+0000", 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068052+0000", 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0299999999999998e-07 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068055+0000", 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5059999999999998e-06 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068063+0000", 2024-09-16T11:14:23.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0369999999999995e-06 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068068+0000", 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4099999999999999e-06 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077461+0000", 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093930579999999993 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077465+0000", 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5250000000000001e-06 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101806 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068886+0000", 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46274375400000001, 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47739746199999999, 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.487 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101806 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068886+0000", 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068886+0000", 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068886+0000", 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.65e-07 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068888+0000", 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.852e-06 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068899+0000", 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0958000000000001e-05 2024-09-16T11:14:23.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068903+0000", 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0940000000000001e-06 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078431+0000", 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095279170000000003 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078435+0000", 2024-09-16T11:14:23.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5829999999999998e-06 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101851 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069636+0000", 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46199399899999999, 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47668111400000002, 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101851 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069636+0000", 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069636+0000", 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069636+0000", 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069638+0000", 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2170000000000001e-06 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069644+0000", 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7620000000000001e-06 2024-09-16T11:14:23.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069648+0000", 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.641e-06 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079392+0000", 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097438490000000006 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079397+0000", 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5999999999999997e-06 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102090 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071213+0000", 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46041658699999999, 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47513544400000002, 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102090 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071213+0000", 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071213+0000", 2024-09-16T11:14:23.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071214+0000", 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3200000000000005e-07 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071216+0000", 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071222+0000", 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9819999999999996e-06 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071226+0000", 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080369+0000", 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091427550000000007 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080372+0000", 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6679999999999999e-06 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102361 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072322+0000", 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45930779700000002, 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47405991600000003, 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.499 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102361 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072322+0000", 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072322+0000", 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072322+0000", 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4999999999999998e-07 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072324+0000", 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.877e-06 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072330+0000", 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7769999999999999e-06 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072333+0000", 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.551e-06 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081462+0000", 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009128677 2024-09-16T11:14:23.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081466+0000", 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.563e-06 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102600 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073023+0000", 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.458606448, 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47339200199999998, 2024-09-16T11:14:23.503 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102600 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073023+0000", 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073023+0000", 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073024+0000", 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073026+0000", 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.125e-06 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073034+0000", 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000005e-06 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073037+0000", 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6849999999999999e-06 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082348+0000", 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093102700000000007 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082351+0000", 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.411e-06 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102644 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073777+0000", 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.457852602, 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47266967100000001, 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.507 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102644 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073777+0000", 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073777+0000", 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073778+0000", 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3099999999999997e-07 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073780+0000", 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.967e-06 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073786+0000", 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8220000000000003e-06 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073789+0000", 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082858+0000", 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090695540000000005 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082862+0000", 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5710000000000002e-06 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101613 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067263+0000", 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46436631099999998, 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.479218164, 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.511 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101613 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067263+0000", 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067263+0000", 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067264+0000", 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0699999999999998e-07 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067266+0000", 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.243e-06 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067272+0000", 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2979999999999997e-06 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067277+0000", 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076634+0000", 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093573590000000009 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076637+0000", 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3519999999999998e-06 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101735 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068073+0000", 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46355703799999998, 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47844087099999999, 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.515 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101735 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068073+0000", 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068073+0000", 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068073+0000", 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1399999999999997e-07 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068075+0000", 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0679999999999999e-06 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068081+0000", 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9460000000000003e-06 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068085+0000", 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.585e-06 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077504+0000", 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094196060000000005 2024-09-16T11:14:23.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9550000000000002e-06 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101808 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068913+0000", 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46271661600000003, 2024-09-16T11:14:23.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47763372599999998, 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101808 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068913+0000", 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068913+0000", 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068913+0000", 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068915+0000", 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4279999999999999e-06 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068918+0000", 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5640000000000001e-06 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068921+0000", 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5770000000000001e-06 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078459+0000", 2024-09-16T11:14:23.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095383789999999996 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078463+0000", 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3380000000000001e-06 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101853 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069652+0000", 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46197811500000002, 2024-09-16T11:14:23.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47692751100000003, 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101853 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069652+0000", 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069652+0000", 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069652+0000", 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6199999999999999e-07 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069654+0000", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0150000000000002e-06 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069658+0000", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2699999999999998e-06 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069661+0000", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.418e-06 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079423+0000", 2024-09-16T11:14:23.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097618340000000005 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079426+0000", 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102092 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071230+0000", 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46039942099999998, 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47538231600000003, 2024-09-16T11:14:23.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102092 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071230+0000", 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071230+0000", 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071231+0000", 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2399999999999998e-07 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071233+0000", 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1770000000000001e-06 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071239+0000", 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7359999999999998e-06 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071242+0000", 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6229999999999999e-06 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080396+0000", 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091538169999999999 2024-09-16T11:14:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080400+0000", 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4130000000000002e-06 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102363 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072337+0000", 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45929230500000001, 2024-09-16T11:14:23.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.474308589, 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102363 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072337+0000", 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072337+0000", 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072338+0000", 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072340+0000", 2024-09-16T11:14:23.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9609999999999999e-06 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072345+0000", 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072349+0000", 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9589999999999997e-06 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081488+0000", 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091385630000000006 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081499+0000", 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0550000000000001e-05 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102601 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073042+0000", 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45858790700000002, 2024-09-16T11:14:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47363678799999998, 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102601 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073042+0000", 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073042+0000", 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073042+0000", 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073044+0000", 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0959999999999999e-06 2024-09-16T11:14:23.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073050+0000", 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1600000000000003e-06 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073054+0000", 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4640000000000002e-06 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082379+0000", 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093254649999999998 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082385+0000", 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.818e-06 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102646 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073793+0000", 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45783675800000001, 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47291871400000002, 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.539 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102646 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073793+0000", 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073793+0000", 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073793+0000", 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.15e-07 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073795+0000", 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1119999999999999e-06 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073801+0000", 2024-09-16T11:14:23.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8270000000000003e-06 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073805+0000", 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.799e-06 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082889+0000", 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090839639999999999 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082892+0000", 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3720000000000001e-06 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101615 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067281+0000", 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46434873999999998, 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47946243, 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101615 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067281+0000", 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067281+0000", 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067282+0000", 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0900000000000001e-07 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067284+0000", 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9970000000000001e-06 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067295+0000", 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0922e-05 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067299+0000", 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2080000000000002e-06 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076658+0000", 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093593489999999994 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076662+0000", 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5700000000000001e-06 2024-09-16T11:14:23.546 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101737 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068097+0000", 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46353269200000002, 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.478679566, 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101737 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068097+0000", 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068097+0000", 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068097+0000", 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068099+0000", 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.787e-06 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068103+0000", 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2729999999999996e-06 2024-09-16T11:14:23.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068106+0000", 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5830000000000001e-06 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0094292339999999999 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077541+0000", 2024-09-16T11:14:23.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5439999999999998e-06 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101810 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068924+0000", 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46270575000000003, 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47788413000000002, 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101810 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068924+0000", 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068924+0000", 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068925+0000", 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6300000000000005e-07 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068926+0000", 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8530000000000001e-06 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068931+0000", 2024-09-16T11:14:23.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4490000000000001e-06 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068936+0000", 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.797e-06 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078489+0000", 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095533690000000008 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078493+0000", 2024-09-16T11:14:23.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6559999999999998e-06 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101855 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069663+0000", 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46196671700000003, 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47717853500000001, 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.555 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101855 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069663+0000", 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069663+0000", 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069663+0000", 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.79e-07 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069665+0000", 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069668+0000", 2024-09-16T11:14:23.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.101e-06 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069670+0000", 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2230000000000001e-06 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079452+0000", 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0097820660000000007 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079456+0000", 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6950000000000002e-06 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102094 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071248+0000", 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46038211000000001, 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47562745699999998, 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102094 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071248+0000", 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071248+0000", 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071248+0000", 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3599999999999999e-07 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071250+0000", 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071255+0000", 2024-09-16T11:14:23.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.519e-06 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071259+0000", 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5190000000000001e-06 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080430+0000", 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091707070000000002 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080433+0000", 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7290000000000002e-06 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102365 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072369+0000", 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.459261111, 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47453831899999999, 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102365 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072369+0000", 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072369+0000", 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072369+0000", 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072372+0000", 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5239999999999999e-06 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072379+0000", 2024-09-16T11:14:23.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4170000000000003e-06 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072383+0000", 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081528+0000", 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091442439999999993 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081531+0000", 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102602 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073060+0000", 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45856961400000001, 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47388040399999998, 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.567 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102602 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073060+0000", 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073060+0000", 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073061+0000", 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0499999999999999e-07 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073063+0000", 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.165e-06 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073069+0000", 2024-09-16T11:14:23.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2219999999999999e-06 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073073+0000", 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7589999999999999e-06 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082411+0000", 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093385859999999994 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082415+0000", 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.354e-06 2024-09-16T11:14:23.570 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102648 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073809+0000", 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45782082099999999, 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47316350000000001, 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.571 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102648 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073809+0000", 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073809+0000", 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073809+0000", 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4099999999999999e-07 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073812+0000", 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.097e-06 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073817+0000", 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8279999999999999e-06 2024-09-16T11:14:23.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073821+0000", 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.968e-06 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082919+0000", 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090972039999999994 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082922+0000", 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6090000000000001e-06 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101616 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.067308+0000", 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46432181700000003, 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.479697665, 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101616 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067308+0000", 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067308+0000", 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067308+0000", 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067311+0000", 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.277e-06 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067318+0000", 2024-09-16T11:14:23.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4610000000000002e-06 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.067322+0000", 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076681+0000", 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0093590199999999991 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.076685+0000", 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101739 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068121+0000", 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46350840100000001, 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47891630600000001, 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.579 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101739 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068121+0000", 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068121+0000", 2024-09-16T11:14:23.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068122+0000", 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068124+0000", 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0509999999999999e-06 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068135+0000", 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1114999999999999e-05 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068139+0000", 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9639999999999997e-06 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077567+0000", 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009427965 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077571+0000", 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4759999999999998e-06 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101812 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.068946+0000", 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.462683872, 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47812491400000001, 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101812 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068946+0000", 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068946+0000", 2024-09-16T11:14:23.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068946+0000", 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6600000000000002e-07 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068948+0000", 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9850000000000001e-06 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068954+0000", 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8390000000000003e-06 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.068958+0000", 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9179999999999996e-06 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078515+0000", 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095573120000000001 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078519+0000", 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3579999999999999e-06 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:101857 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.069672+0000", 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.46195770000000003, 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47743354100000002, 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 101857 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069672+0000", 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069672+0000", 2024-09-16T11:14:23.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069672+0000", 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6e-07 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069674+0000", 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2550000000000001e-06 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069677+0000", 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0639999999999998e-06 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.069679+0000", 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4880000000000001e-06 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079482+0000", 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0098025339999999999 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.079485+0000", 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4520000000000002e-06 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102095 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.071278+0000", 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.460351548, 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47585935800000001, 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.591 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102095 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071278+0000", 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071278+0000", 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071279+0000", 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7000000000000004e-07 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071286+0000", 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9859999999999997e-06 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071293+0000", 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1620000000000002e-06 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.071297+0000", 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9840000000000003e-06 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080456+0000", 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091594869999999991 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.080460+0000", 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4290000000000001e-06 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102368 4.d 4.1ff161ed (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.072403+0000", 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45922661599999998, 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47477107099999999, 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102368 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072403+0000", 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072403+0000", 2024-09-16T11:14:23.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072404+0000", 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4700000000000002e-07 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072406+0000", 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.418e-06 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072418+0000", 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2237e-05 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.072430+0000", 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2170999999999999e-05 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081557+0000", 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091271110000000003 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.081563+0000", 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6300000000000003e-06 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102655 4.d 4.46c3b9cd (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073825+0000", 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.45780432100000001, 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.47338096800000001, 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102655 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073825+0000", 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073825+0000", 2024-09-16T11:14:23.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073826+0000", 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.84e-07 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073828+0000", 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1040000000000001e-06 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073834+0000", 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8799999999999996e-06 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073838+0000", 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8199999999999998e-06 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082949+0000", 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091115699999999994 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.082953+0000", 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.473e-06 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 256 2024-09-16T11:14:23.603 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:23.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:23.605 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:23.599+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:23.605 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:23.706 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:23.703+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:23.712 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:23.713 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:23.722 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:23.735 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:23.755 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:23.807 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:23.801+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.738853384, 2024-09-16T11:14:23.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:23.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:14:23.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.822 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7388340179999999, 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:23.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:14:23.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7387932560000001, 2024-09-16T11:14:23.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:23.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.833 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7387624580000001, 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:23.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.838 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.73874387, 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.839 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:23.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:23.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:14:23.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.738725818, 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.844 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.848 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7387058909999999, 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:23.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:23.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:23.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:14:23.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.738687052, 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:14:23.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.858 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.738495103, 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:23.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:23.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:14:23.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.863 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7384758229999999, 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:14:23.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7350055580000001, 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.869 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:23.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7349842550000001, 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:14:23.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7349741839999999, 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:14:23.879 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:23.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:14:23.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.734945881, 2024-09-16T11:14:23.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:23.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.889 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7349319520000002, 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:14:23.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7349231889999999, 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:23.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:14:23.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7348993699999999, 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:14:23.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:14:23.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:14:23.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7348682479999999, 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.905 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:14:23.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:14:23.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.909 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.734820509, 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:23.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.914 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 1.7347984009999999, 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:14:23.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:23.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:23.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:23.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:23.919 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:23.921 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:23.921 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:23.913+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:24.022 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:24.013+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:24.026 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:24.026 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:24.035 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:24.075 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:24.075 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:24.075 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T11:14:24.076 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:14:24.076 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:14:24.076 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-16T11:14:24.101 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:14:24.122 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:24.117+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:24.141 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:24.172 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:24.205 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:14:24.205 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:24.206 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:24.206 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:24.207 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:24.218 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:24.236 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:24.262 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:24.257+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:24.266 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:24.291 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:24.294 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:24.295 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:24.324 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:24.319+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:24.327 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:24.327 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:24.336 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:24.425 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:24.417+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:24.429 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:24.429 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:24.438 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:24.525 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:24.523+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:24.557 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:24.557 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:24.568 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:24.626 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:24.621+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:24.673 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:24.673 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:24.681 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:24.727 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:24.723+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:24.780 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:24.781 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:24.790 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:24.829 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:24.827+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:24.923 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:24.924 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:24.930 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:24.927+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:24.934 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:25.031 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:25.027+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:25.045 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.046 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.057 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:25.115 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:14:25.131 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:25.125+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:25.170 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.171 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.182 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:25.232 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:25.225+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:25.290 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.291 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.301 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:25.333 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:25.331+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:25.423 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.423 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:25.434 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:25.431+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:25.531 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.531 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.535 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:25.531+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:25.540 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:25.598 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:25.616 INFO:teuthology.orchestra.run.smithi162.stdout:3271 write 0xa828c96 thru 0xa82c5e8 (0x3953 bytes) 2024-09-16T11:14:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:25.621 INFO:teuthology.orchestra.run.smithi162.stdout:3273 write 0x657d83b thru 0x658129b (0x3a61 bytes) 2024-09-16T11:14:25.636 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:25.636 INFO:teuthology.orchestra.run.smithi162.stdout:3275 punch from 0x2a50057 to 0x2a5dc60, (0xdc09 bytes) 2024-09-16T11:14:25.636 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:25.631+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:25.644 INFO:teuthology.orchestra.run.smithi162.stdout:3276 trunc from 0xa82c5e9 to 0x4ed29d4 2024-09-16T11:14:25.702 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.702 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.709 INFO:teuthology.orchestra.run.smithi162.stdout:3277 punch from 0x1dc195b to 0x1dc3b96, (0x223b bytes) 2024-09-16T11:14:25.709 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:25.709 INFO:teuthology.orchestra.run.smithi162.stdout:3279 writesame 0x35936b9 thru 0x3598dfc data_size 0xa(0x5744 bytes) 2024-09-16T11:14:25.712 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:25.737 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:25.735+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:25.741 INFO:teuthology.orchestra.run.smithi162.stdout:3280 write 0xadf24e3 thru 0xadfffa8 (0xdac6 bytes) 2024-09-16T11:14:25.746 INFO:teuthology.orchestra.run.smithi162.stdout:3281 write 0x581678d thru 0x58176ba (0xf2e bytes) 2024-09-16T11:14:25.750 INFO:teuthology.orchestra.run.smithi162.stdout:3283 read 0x27a8adf thru 0x27b541c (0xc93e bytes) 2024-09-16T11:14:25.771 INFO:teuthology.orchestra.run.smithi162.stdout:3284 writesame 0xc561d1d thru 0xc564ba7 data_size 0x1(0x2e8b bytes) 2024-09-16T11:14:25.772 INFO:teuthology.orchestra.run.smithi162.stdout:3285 read 0xf39ae5 thru 0xf39cff (0x21b bytes) 2024-09-16T11:14:25.772 INFO:teuthology.orchestra.run.smithi162.stdout:3286 clone 27 order 19 su 32768 sc 3 2024-09-16T11:14:25.816 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.816 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.825 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:25.869 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:25.867+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:25.939 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:25.933+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:25.974 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:25.974 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:25.983 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:26.040 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:26.033+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:26.079 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:26.080 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:26.089 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:26.141 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:26.133+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:26.194 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:26.194 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:26.204 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:26.239 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone26 from 0xc564ba8 (overlap 0x4ed29d4) to 0x0 2024-09-16T11:14:26.242 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:26.237+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:26.306 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:26.322 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:26.322 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:26.336 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:26.343 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:26.337+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:26.444 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:26.439+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:26.453 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:26.457 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:26.457 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:26.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:26.477 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:26.496 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:26.515 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:26.524 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-16T11:14:26.545 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:26.545 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:26.537+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49775 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1286144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.432167616999998, 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050091745, 2024-09-16T11:14:26.570 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49775 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133155+0000", 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3900000000000001e-07 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133156+0000", 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1179999999999999e-06 2024-09-16T11:14:26.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133159+0000", 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133178+0000", 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8692e-05 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.182940+0000", 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049762038000000001 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183007+0000", 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7082000000000006e-05 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183247+0000", 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000239581 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.432142681000002, 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.432008938000003, 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.580 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:26.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.431990550000002, 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.584 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:26.588 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.431689792999997, 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:26.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:26.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.431660088000001, 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:26.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.431646690000001, 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.597 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:26.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:26.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.601 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.431632886000003, 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:26.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.431257887000001, 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.606 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:26.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:26.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.417713855999999, 2024-09-16T11:14:26.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.417692281999997, 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:26.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:26.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:26.619 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.417675783, 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:26.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.623 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.417659706999999, 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:26.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:26.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.417276133000001, 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:26.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50292 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1630208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.632 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.405213234999998, 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050216150000000001, 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50292 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160109+0000", 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160110+0000", 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160117+0000", 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.362e-06 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160124+0000", 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4439999999999996e-06 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160126+0000", 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:14:26.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210014+0000", 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049887881000000002 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210108+0000", 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3356000000000005e-05 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210325+0000", 2024-09-16T11:14:26.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021782400000000001 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.405193955999998, 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:26.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.405185754999998, 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:26.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:26.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.405177733000002, 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.646 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:26.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:26.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:26.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 9.7216988539999996, 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:14:26.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:14:26.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:14:26.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.4882805030000004, 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:14:26.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:26.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:14:26.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:26.662 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:26.662 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:26.663 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:26.664 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:26.664 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:26.664 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:26.665 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:26.665 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:26.665 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:26.665 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:26.665 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:26.665 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:26.666 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:26.666 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:26.666 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:26.659+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:26.764 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:26.759+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:26.771 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:26.771 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:26.771 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:26.771 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:26.780 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:26.865 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:26.857+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:26.881 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:26.881 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:26.881 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:26.881 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:26.881 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:26.890 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:26.919 INFO:teuthology.orchestra.run.smithi162.stdout:3287 read 0x55cf528 thru 0x55d5f06 (0x69df bytes) 2024-09-16T11:14:26.929 INFO:teuthology.orchestra.run.smithi162.stdout:3289 write 0x6a44ca2 thru 0x6a4ea78 (0x9dd7 bytes) 2024-09-16T11:14:26.940 INFO:teuthology.orchestra.run.smithi162.stdout:3290 write 0x2bafb9d thru 0x2bb125b (0x16bf bytes) 2024-09-16T11:14:26.951 INFO:teuthology.orchestra.run.smithi162.stdout:3291 write 0x328aeb0 thru 0x3290f3d (0x608e bytes) 2024-09-16T11:14:26.963 INFO:teuthology.orchestra.run.smithi162.stdout:3292 read 0x3c61875 thru 0x3c65f09 (0x4695 bytes) 2024-09-16T11:14:26.964 INFO:teuthology.orchestra.run.smithi162.stdout:3294 read 0x7b3c397 thru 0x7b42968 (0x65d2 bytes) 2024-09-16T11:14:26.964 INFO:teuthology.orchestra.run.smithi162.stdout:3298 write 0x5764eff thru 0x57657fc (0x8fe bytes) 2024-09-16T11:14:26.966 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:26.963+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:26.973 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:26.973 INFO:teuthology.orchestra.run.smithi162.stdout:3301 read 0x62699ad thru 0x627439e (0xa9f2 bytes) 2024-09-16T11:14:26.974 INFO:teuthology.orchestra.run.smithi162.stdout:3302 punch from 0xba932f6 to 0xba9b6ec, (0x83f6 bytes) 2024-09-16T11:14:26.974 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:26.974 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:26.974 INFO:teuthology.orchestra.run.smithi162.stdout:3305 read 0x1a6a44e thru 0x1a6dd4f (0x3902 bytes) 2024-09-16T11:14:26.991 INFO:teuthology.orchestra.run.smithi162.stdout:3306 trunc from 0xc564ba8 to 0x3391c44 2024-09-16T11:14:27.006 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.006 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:27.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:27.006 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:27.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.006 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9292768120000003, 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:27.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.011 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9292574460000003, 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.012 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:27.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:27.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.929216684, 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.017 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:27.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:27.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:27.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.929185886, 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:27.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:14:27.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9291672980000003, 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.027 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:27.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9291492459999997, 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:14:27.032 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:27.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:27.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:14:27.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:27.037 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9291293190000003, 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:27.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:27.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:14:27.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9291104800000003, 2024-09-16T11:14:27.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:27.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:27.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:27.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:27.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:27.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:14:27.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9289185309999999, 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:14:27.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9288992509999998, 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:14:27.054 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:27.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:14:27.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.925428986, 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:27.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:27.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9254076829999995, 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.064 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:14:27.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:27.069 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9253976120000003, 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:27.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.074 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9253693089999997, 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:27.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:14:27.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9253553800000001, 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:27.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:27.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9253466169999998, 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:27.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:27.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:14:27.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9253227979999998, 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:14:27.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:14:27.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9252916759999996, 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.095 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:14:27.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:14:27.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9252439370000003, 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:14:27.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 4.9252218289999998, 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:14:27.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.109 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:27.112 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:27.112 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:27.105+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3311 read 0x2d0ae00 thru 0x2d18781 (0xd982 bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3312 read 0x3e3560 thru 0x3ee694 (0xb135 bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3313 write 0xbbaa133 thru 0xbbacac0 (0x298e bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3315 write 0x4850f64 thru 0x485cb71 (0xbc0e bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3316 punch from 0x146c44e to 0x1472608, (0x61ba bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3317 write 0x37686a1 thru 0x376c6be (0x401e bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3318 punch from 0x3cece76 to 0x3cf5da3, (0x8f2d bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3319 write 0x733bc81 thru 0x73401a2 (0x4522 bytes) 2024-09-16T11:14:27.113 INFO:teuthology.orchestra.run.smithi162.stdout:3320 writesame 0x376f542 thru 0x3779598 data_size 0x1(0xa057 bytes) 2024-09-16T11:14:27.114 INFO:teuthology.orchestra.run.smithi162.stdout:3322 write 0xbad0f63 thru 0xbad1ca9 (0xd47 bytes) 2024-09-16T11:14:27.119 INFO:teuthology.orchestra.run.smithi162.stdout:3323 writesame 0x1f4f158 thru 0x1f5095b data_size 0x1d(0x1804 bytes) 2024-09-16T11:14:27.127 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:14:27.135 INFO:teuthology.orchestra.run.smithi162.stdout:3324 trunc from 0xbbacac1 to 0x9d21aeb 2024-09-16T11:14:27.153 INFO:teuthology.orchestra.run.smithi162.stdout:3325 writesame 0x9a518e8 thru 0x9a54c3e data_size 0xd(0x3357 bytes) 2024-09-16T11:14:27.159 INFO:teuthology.orchestra.run.smithi162.stdout:3326 read 0x3ae2878 thru 0x3aee7b6 (0xbf3f bytes) 2024-09-16T11:14:27.166 INFO:teuthology.orchestra.run.smithi162.stdout:3327 trunc from 0x9d21aeb to 0x7199e32 2024-09-16T11:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:3328 read 0x5387abc thru 0x538fe79 (0x83be bytes) 2024-09-16T11:14:27.187 INFO:teuthology.orchestra.run.smithi162.stdout:3329 read 0x63cd4a7 thru 0x63dc919 (0xf473 bytes) 2024-09-16T11:14:27.192 INFO:teuthology.orchestra.run.smithi162.stdout:3331 trunc from 0x7199e32 to 0x94abbbe 2024-09-16T11:14:27.202 INFO:teuthology.orchestra.run.smithi162.stdout:3332 write 0x4d87e9 thru 0x4e60d4 (0xd8ec bytes) 2024-09-16T11:14:27.211 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:27.205+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:27.216 INFO:teuthology.orchestra.run.smithi162.stdout:3333 write 0x869057 thru 0x86b630 (0x25da bytes) 2024-09-16T11:14:27.226 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.226 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:27.226 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:27.226 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:27.226 INFO:teuthology.orchestra.run.smithi162.stdout:3337 punch from 0x830a4c8 to 0x8315901, (0xb439 bytes) 2024-09-16T11:14:27.236 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:27.237 INFO:teuthology.orchestra.run.smithi162.stdout:3338 write 0xa2b343f thru 0xa2c31c6 (0xfd88 bytes) 2024-09-16T11:14:27.245 INFO:teuthology.orchestra.run.smithi162.stdout:3340 write 0x89c20c8 thru 0x89c614a (0x4083 bytes) 2024-09-16T11:14:27.252 INFO:teuthology.orchestra.run.smithi162.stdout:3341 trunc from 0xa2c31c7 to 0x8b94b66 2024-09-16T11:14:27.292 INFO:teuthology.orchestra.run.smithi162.stdout:3344 write 0xe3a20a3 thru 0xe3ad7f1 (0xb74f bytes) 2024-09-16T11:14:27.301 INFO:teuthology.orchestra.run.smithi162.stdout:3345 read 0xb87f12 thru 0xb8aa6a (0x2b59 bytes) 2024-09-16T11:14:27.304 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:27.304 INFO:teuthology.orchestra.run.smithi162.stdout:3349 read 0x5f101d0 thru 0x5f1a6c5 (0xa4f6 bytes) 2024-09-16T11:14:27.304 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:27.304 INFO:teuthology.orchestra.run.smithi162.stdout:3351 writesame 0x7ebbd00 thru 0x7ebff4e data_size 0x23(0x424f bytes) 2024-09-16T11:14:27.311 INFO:teuthology.orchestra.run.smithi162.stdout:3352 writesame 0x4983544 thru 0x4990326 data_size 0x1(0xcde3 bytes) 2024-09-16T11:14:27.312 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:27.305+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:27.338 INFO:teuthology.orchestra.run.smithi162.stdout:3353 write 0x6b53ee4 thru 0x6b55d30 (0x1e4d bytes) 2024-09-16T11:14:27.343 INFO:teuthology.orchestra.run.smithi162.stdout:3354 write 0xcf9ef91 thru 0xcfab17e (0xc1ee bytes) 2024-09-16T11:14:27.351 INFO:teuthology.orchestra.run.smithi162.stdout:3355 write 0x71719db thru 0x7180cd7 (0xf2fd bytes) 2024-09-16T11:14:27.360 INFO:teuthology.orchestra.run.smithi162.stdout:3356 writesame 0x9eb37ea thru 0x9eb4ae5 data_size 0x36(0x12fc bytes) 2024-09-16T11:14:27.364 INFO:teuthology.orchestra.run.smithi162.stdout:3357 read 0xd5e1269 thru 0xd5e3081 (0x1e19 bytes) 2024-09-16T11:14:27.365 INFO:teuthology.orchestra.run.smithi162.stdout:3358 punch from 0xe208499 to 0xe20b277, (0x2dde bytes) 2024-09-16T11:14:27.365 INFO:teuthology.orchestra.run.smithi162.stdout:3359 read 0x5e3663d thru 0x5e458f5 (0xf2b9 bytes) 2024-09-16T11:14:27.365 INFO:teuthology.orchestra.run.smithi162.stdout:3361 write 0x29f2a9b thru 0x29fed0a (0xc270 bytes) 2024-09-16T11:14:27.376 INFO:teuthology.orchestra.run.smithi162.stdout:3362 read 0x12d783b thru 0x12e4a7e (0xd244 bytes) 2024-09-16T11:14:27.378 INFO:teuthology.orchestra.run.smithi162.stdout:3363 write 0x3f224f7 thru 0x3f25782 (0x328c bytes) 2024-09-16T11:14:27.389 INFO:teuthology.orchestra.run.smithi162.stdout:3366 writesame 0x25d5853 thru 0x25e2bd3 data_size 0x77(0xd381 bytes) 2024-09-16T11:14:27.400 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.400 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:27.400 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:27.400 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:27.400 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:27.408 INFO:teuthology.orchestra.run.smithi162.stdout:3367 read 0xae64133 thru 0xae696bf (0x558d bytes) 2024-09-16T11:14:27.408 INFO:teuthology.orchestra.run.smithi162.stdout:3370 write 0xab85be5 thru 0xab8c775 (0x6b91 bytes) 2024-09-16T11:14:27.410 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:27.413 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:27.407+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:27.414 INFO:teuthology.orchestra.run.smithi162.stdout:3372 punch from 0x958f5bf to 0x9595afd, (0x653e bytes) 2024-09-16T11:14:27.414 INFO:teuthology.orchestra.run.smithi162.stdout:3373 clone 28 order 23 su 2097152 sc 4 2024-09-16T11:14:27.507 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.641216115999995, 2024-09-16T11:14:27.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:27.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.513 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473244467000001, 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:27.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:27.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473232906, 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.518 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:27.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:27.522 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473219704000002, 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:27.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:27.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473207264000003, 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.527 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:27.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.531 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473193752999997, 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:27.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:27.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:27.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473182838, 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.536 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:27.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.540 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473156920000001, 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:27.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:27.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473141466000001, 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:27.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:27.549 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.47313158, 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:27.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473121816999999, 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:27.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:27.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:27.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:27.558 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473111639999999, 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:27.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:27.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473095158, 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:27.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473080461000002, 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:27.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:27.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473062018, 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:27.572 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:27.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473045069000001, 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:27.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473025856, 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:27.581 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:27.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:27.585 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.473008640000003, 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.586 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:27.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.590 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.472987386, 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:27.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.472967418000003, 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.595 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:27.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.599 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:27.601 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:27.601 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:27.597+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:27.701 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:27.693+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:27.752 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.752 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:27.752 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:27.752 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:27.762 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:27.806 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:27.801+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:27.865 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.868 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:27.868 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:27.868 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:27.868 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:27.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:27.902 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:27.899+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.789466134000001, 2024-09-16T11:14:27.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:27.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:27.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.785107607, 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:27.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.781469076999997, 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:27.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:27.993 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.781450290999999, 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:27.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:27.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:27.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:27.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:27.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:27.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:27.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:27.999 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.781432967999997, 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:28.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.781415498999998, 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.004 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.008 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777575040000002, 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:28.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:28.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:28.013 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777564925, 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777555044000003, 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:28.018 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:28.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.022 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777545867999997, 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:28.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:28.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:28.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.027 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777529801, 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:28.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:28.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:28.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777513405999997, 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777503832000001, 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:28.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:28.041 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.777494296, 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.042 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50319 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.773447902000001, 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118443188, 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.046 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50319 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201802+0000", 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201804+0000", 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.342e-06 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201807+0000", 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9490000000000001e-06 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201809+0000", 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2299999999999998e-06 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320222+0000", 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118412609 2024-09-16T11:14:28.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320245+0000", 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3805999999999999e-05 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.773437639000001, 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:28.050 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:28.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.773428256999999, 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:28.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.773419599, 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:28.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:28.062 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.773410364999997, 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:28.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.079657413, 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.067 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:14:28.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.071 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.072 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:28.076 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:28.076 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:28.065+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:28.134 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:14:28.173 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:28.165+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:28.187 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:28.187 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:28.187 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:28.187 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:28.197 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:28.258 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone27 from 0xe3ad7f2 (overlap 0x3391c44) to 0x1ef10f5 2024-09-16T11:14:28.274 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:28.271+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:28.304 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:28.304 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:28.304 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:28.304 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:28.304 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:28.315 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:28.375 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:28.371+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:28.418 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.638184637999998, 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:28.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:28.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:28.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.250258938000002, 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:28.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.250107607999993, 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:28.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.250056122000004, 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:28.438 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:28.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:28.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.249985973999998, 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:28.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.249813504000002, 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:28.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:28.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:28.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.249755182000001, 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:28.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.249603062000006, 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.462 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:28.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:28.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:28.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:28.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.249424543000003, 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:28.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:28.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:28.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 98.24939157, 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.474 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:28.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:28.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:28.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 97.697203845000004, 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:28.481 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:28.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:28.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:28.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:28.487 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 97.697075048000002, 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:28.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.493 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 97.697016703000003, 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:28.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 97.696962314000004, 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:28.500 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:28.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.506 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 97.696828006999993, 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:28.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:28.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.512 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 96.915672638000004, 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.513 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:28.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:28.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 96.915585325999999, 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.519 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:28.524 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 96.915347198000006, 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:28.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:28.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 26.234967188999999, 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:28.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 11.526774753, 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:28.535 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:14:28.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:14:28.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:14:28.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:28.539 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:28.541 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-16T11:14:28.541 INFO:teuthology.orchestra.run.smithi162.stdout:3374 writesame 0x58e2e43 thru 0x58e7334 data_size 0xa(0x44f2 bytes) 2024-09-16T11:14:28.541 INFO:teuthology.orchestra.run.smithi162.stdout:3375 read 0x83d9773 thru 0x83e2b91 (0x941f bytes) 2024-09-16T11:14:28.541 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:28.541 INFO:teuthology.orchestra.run.smithi162.stdout:3377 clone 29 order 24 su 4194304 sc 13 2024-09-16T11:14:28.541 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:28.535+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:28.542 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:28.642 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:28.639+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:28.651 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:28.652 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:28.661 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:28.666 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:28.743 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:28.737+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:28.761 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:28.762 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:28.762 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:28.762 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:28.762 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:28.772 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:28.798 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:28.819 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:28.838 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:28.844 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:28.841+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:28.858 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:28.866 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:28.866 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:28.875 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:28.881 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:28.911 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:28.944 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:28.939+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:28.945 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:28.972 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:28.975 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:28.976 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:28.985 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:28.985 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:28.994 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:29.045 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:29.041+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:29.144 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:29.144 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:29.145 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:29.143+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:29.154 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:29.218 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:29.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:29.218 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 3, 4, 5, 6, 7, 2] 2024-09-16T11:14:29.218 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-16T11:14:29.247 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:29.241+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:29.251 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:29.252 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:29.262 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:29.263 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone28 from 0xe3ad7f2 (overlap 0xe3ad7f2) to 0x2d89196 2024-09-16T11:14:29.348 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:29.341+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:29.386 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:29.386 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:29.401 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:29.449 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:29.447+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:29.520 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-16T11:14:29.538 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:29.538 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:29.550 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:29.545+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:29.553 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:29.651 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:29.647+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:29.654 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:14:29.651+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e86 definitely_dead 0 2024-09-16T11:14:29.698 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:29.698 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:29.708 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:29.741 INFO:teuthology.orchestra.run.smithi162.stdout:3378 punch from 0x7ee8a75 to 0x7ee99e9, (0xf74 bytes) 2024-09-16T11:14:29.748 INFO:teuthology.orchestra.run.smithi162.stdout:3379 writesame 0xd921a7 thru 0xd9b4fb data_size 0x1(0x9355 bytes) 2024-09-16T11:14:29.753 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:29.745+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:29.833 INFO:teuthology.orchestra.run.smithi162.stdout:3380 write 0x9ac2347 thru 0x9ac95e1 (0x729b bytes) 2024-09-16T11:14:29.841 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:29.842 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:29.852 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:29.876 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:29.876 INFO:teuthology.orchestra.run.smithi162.stdout:3384 read 0x8fb1bec thru 0x8fb20e3 (0x4f8 bytes) 2024-09-16T11:14:29.877 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:29.871+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:29.878 INFO:teuthology.orchestra.run.smithi162.stdout:3385 read 0xb0e2479 thru 0xb0eb93e (0x94c6 bytes) 2024-09-16T11:14:29.878 INFO:teuthology.orchestra.run.smithi162.stdout:3386 read 0x3181ba6 thru 0x318dd2b (0xc186 bytes) 2024-09-16T11:14:29.887 INFO:teuthology.orchestra.run.smithi162.stdout:3389 read 0x35da3d4 thru 0x35dbaea (0x1717 bytes) 2024-09-16T11:14:29.889 INFO:teuthology.orchestra.run.smithi162.stdout:3390 write 0xc0444b3 thru 0xc052aca (0xe618 bytes) 2024-09-16T11:14:29.954 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:29.949+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:29.967 INFO:teuthology.orchestra.run.smithi162.stdout:3391 write 0x1033408 thru 0x10425a1 (0xf19a bytes) 2024-09-16T11:14:30.000 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.000 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.011 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:30.034 INFO:teuthology.orchestra.run.smithi162.stdout:3392 writesame 0x8d7ffb8 thru 0x8d8ca1e data_size 0x2b(0xca67 bytes) 2024-09-16T11:14:30.038 INFO:teuthology.orchestra.run.smithi162.stdout:3393 write 0x8eecd70 thru 0x8efbbad (0xee3e bytes) 2024-09-16T11:14:30.042 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:30.042 INFO:teuthology.orchestra.run.smithi162.stdout:3395 punch from 0xbc7563d to 0xbc845fb, (0xefbe bytes) 2024-09-16T11:14:30.042 INFO:teuthology.orchestra.run.smithi162.stdout:3396 punch from 0x4ee65d7 to 0x4ee7494, (0xebd bytes) 2024-09-16T11:14:30.055 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:30.053+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:30.061 INFO:teuthology.orchestra.run.smithi162.stdout:3397 trunc from 0xe3ad7f2 to 0x30d1d25 2024-09-16T11:14:30.132 INFO:teuthology.orchestra.run.smithi162.stdout:3398 read 0x355109 thru 0x357238 (0x2130 bytes) 2024-09-16T11:14:30.148 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:14:30.154 INFO:teuthology.orchestra.run.smithi162.stderr:marked out osd.4. 2024-09-16T11:14:30.156 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:30.151+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:30.161 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.161 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.171 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:30.258 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:30.255+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:30.278 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.278 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.288 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:30.359 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:30.353+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:30.391 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.392 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.404 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:30.460 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:30.453+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:30.502 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.503 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.512 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:30.561 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:30.553+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:30.656 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.657 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.662 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:30.659+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:30.665 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:30.763 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:30.757+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:30.763 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.764 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.773 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:30.865 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:30.859+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:30.868 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.868 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:30.965 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:30.963+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:30.978 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:30.978 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:30.985 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:30.988 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:31.066 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:31.063+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:31.081 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:31.082 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:31.091 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:31.128 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:31.149 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:31.167 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:31.161+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:31.167 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:31.186 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:31.190 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:31.190 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:31.199 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:31.208 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:31.228 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:31.247 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:31.268 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:31.263+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:31.268 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:31.271 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:31.271 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:31.271 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:31.271 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:31.272 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:31.272 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:31.272 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:31.272 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:31.295 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:31.295 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:31.296 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:31.296 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:31.305 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:31.368 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:31.363+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:31.401 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:31.402 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:31.402 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:31.402 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:31.402 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:31.410 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:31.443 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:14:31.468 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:31.461+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:31.529 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:31.529 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:31.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.74938529799999, 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:31.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:31.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:31.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:31.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:31.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:31.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.361459598, 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.537 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.361308268, 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:31.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:31.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.548 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.361256782, 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:31.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.36118663400001, 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:31.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.361014164, 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.563 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:31.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:31.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:31.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.360955842, 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.569 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:31.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.574 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.360803722, 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:31.580 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.360625203, 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:31.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:31.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:31.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:31.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 101.36059222999999, 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:31.586 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:31.591 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.808404505, 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:31.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:31.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.808275708, 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:31.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:31.603 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.808217363, 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:31.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:31.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.808162974, 2024-09-16T11:14:31.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:31.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.808028667, 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:31.615 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:31.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:31.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.026873298, 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:31.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:31.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.626 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.02678598599999, 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.632 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.026547858, 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:31.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.346167849, 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.638 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:31.642 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.637975412999999, 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:14:31.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:14:31.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:31.646 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:31.649 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:3400 write 0xad8e701 thru 0xad8fc35 (0x1535 bytes) 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:3405 punch from 0x91b1b3b to 0x91ba671, (0x8b36 bytes) 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:3406 read 0x351aea5 thru 0x3520bb7 (0x5d13 bytes) 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:3407 read 0xe95100 thru 0xe977da (0x26db bytes) 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:3408 write 0xe48d52f thru 0xe48dd5b (0x82d bytes) 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:3410 punch from 0x772288f to 0x772ba42, (0x91b3 bytes) 2024-09-16T11:14:31.649 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:31.650 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:31.650 INFO:teuthology.orchestra.run.smithi162.stdout:3415 read 0x6845e6d thru 0x6854201 (0xe395 bytes) 2024-09-16T11:14:31.650 INFO:teuthology.orchestra.run.smithi162.stdout:3416 writesame 0x2fecc0a thru 0x2ff04e8 data_size 0x17(0x38df bytes) 2024-09-16T11:14:31.650 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:31.643+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:31.749 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:31.747+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:31.769 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:31.769 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:31.769 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:31.769 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:31.780 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:31.849 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:31.847+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:31.945 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:31.946 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:31.946 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:31.946 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:31.946 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:31.950 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:31.945+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:31.956 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:32.052 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:32.047+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.894227125, 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.073 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.077 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.894149382000002, 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.078 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:32.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.891783367000002, 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:32.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:32.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:32.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.891763851, 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:32.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:32.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.890626347000001, 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:32.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:32.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.890610447999997, 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:32.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:32.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.890586194000001, 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:32.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:32.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:32.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.890570935, 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.890555616, 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:32.116 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888533211999999, 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:32.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:32.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:32.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:32.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:32.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:32.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:32.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:32.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:32.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:32.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888518714, 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:32.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:32.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888504380999997, 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.130 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:32.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888490324000003, 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:32.134 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:32.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.138 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888476468, 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:32.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888462593, 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888448826000001, 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.152 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888401317000003, 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.888115499000001, 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.157 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:32.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:32.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:32.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.795451683000003, 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:32.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:32.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:32.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:32.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:32.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:32.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:32.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:32.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:32.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:32.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.795302501000002, 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:32.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:32.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:32.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:32.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:32.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:32.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:32.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.180 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:32.181 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:32.182 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:32.173+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:32.230 INFO:teuthology.orchestra.run.smithi162.stdout:3417 writesame 0x270e381 thru 0x27126eb data_size 0x3(0x436b bytes) 2024-09-16T11:14:32.239 INFO:teuthology.orchestra.run.smithi162.stdout:3418 read 0xa0cc063 thru 0xa0d1b12 (0x5ab0 bytes) 2024-09-16T11:14:32.240 INFO:teuthology.orchestra.run.smithi162.stdout:3419 punch from 0xcab9e1a to 0xcac0137, (0x631d bytes) 2024-09-16T11:14:32.241 INFO:teuthology.orchestra.run.smithi162.stdout:3420 read 0x4243c70 thru 0x4244daf (0x1140 bytes) 2024-09-16T11:14:32.262 INFO:teuthology.orchestra.run.smithi162.stdout:3421 trunc from 0xe48dd5c to 0x3b51bfe 2024-09-16T11:14:32.282 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:32.277+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:32.300 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:32.300 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:32.300 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:32.300 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:32.314 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:32.316 INFO:teuthology.orchestra.run.smithi162.stdout:3422 write 0xca0f717 thru 0xca13419 (0x3d03 bytes) 2024-09-16T11:14:32.319 INFO:teuthology.orchestra.run.smithi162.stdout:3424 read 0x77b8af7 thru 0x77ba5ad (0x1ab7 bytes) 2024-09-16T11:14:32.322 INFO:teuthology.orchestra.run.smithi162.stdout:3426 write 0x1a0a50f thru 0x1a0d9be (0x34b0 bytes) 2024-09-16T11:14:32.326 INFO:teuthology.orchestra.run.smithi162.stdout:3428 punch from 0x8edaf35 to 0x8edd5b5, (0x2680 bytes) 2024-09-16T11:14:32.326 INFO:teuthology.orchestra.run.smithi162.stdout:3429 write 0x8fb6944 thru 0x8fb9acd (0x318a bytes) 2024-09-16T11:14:32.328 INFO:teuthology.orchestra.run.smithi162.stdout:3430 writesame 0xc77e6f4 thru 0xc78cef9 data_size 0x11(0xe806 bytes) 2024-09-16T11:14:32.353 INFO:teuthology.orchestra.run.smithi162.stdout:3431 trunc from 0xca1341a to 0xfdb009 2024-09-16T11:14:32.383 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:32.377+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:32.419 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:32.419 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:32.419 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:32.419 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:32.419 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:32.431 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:32.435 INFO:teuthology.orchestra.run.smithi162.stdout:3432 write 0x78d7fdc thru 0x78e2b8c (0xabb1 bytes) 2024-09-16T11:14:32.447 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.447 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.447 INFO:teuthology.orchestra.run.smithi162.stdout:3436 write 0xc1093e thru 0xc1a11b (0x97de bytes) 2024-09-16T11:14:32.466 INFO:teuthology.orchestra.run.smithi162.stdout:3437 trunc from 0x78e2b8d to 0xde7c50e 2024-09-16T11:14:32.477 INFO:teuthology.orchestra.run.smithi162.stdout:3439 read 0xd453082 thru 0xd454dbf (0x1d3e bytes) 2024-09-16T11:14:32.481 INFO:teuthology.orchestra.run.smithi162.stdout:3440 trunc from 0xde7c50e to 0xc39ee9c 2024-09-16T11:14:32.484 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:32.477+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:32.491 INFO:teuthology.orchestra.run.smithi162.stdout:3441 punch from 0x7d8bdd8 to 0x7d991c3, (0xd3eb bytes) 2024-09-16T11:14:32.491 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.491 INFO:teuthology.orchestra.run.smithi162.stdout:3445 writesame 0x63ee43 thru 0x64a930 data_size 0x47(0xbaee bytes) 2024-09-16T11:14:32.498 INFO:teuthology.orchestra.run.smithi162.stdout:3446 punch from 0x42da244 to 0x42e2ddc, (0x8b98 bytes) 2024-09-16T11:14:32.498 INFO:teuthology.orchestra.run.smithi162.stdout:3447 punch from 0x8c15bdb to 0x8c1e41f, (0x8844 bytes) 2024-09-16T11:14:32.498 INFO:teuthology.orchestra.run.smithi162.stdout:3448 write 0x14d34d3 thru 0x14e2a1b (0xf549 bytes) 2024-09-16T11:14:32.508 INFO:teuthology.orchestra.run.smithi162.stdout:3449 punch from 0x7fa64f to 0x7fb51f, (0xed0 bytes) 2024-09-16T11:14:32.508 INFO:teuthology.orchestra.run.smithi162.stdout:3450 read 0x1041d0d thru 0x1049444 (0x7738 bytes) 2024-09-16T11:14:32.509 INFO:teuthology.orchestra.run.smithi162.stdout:3452 writesame 0x59ff14d thru 0x5a061fa data_size 0x1(0x70ae bytes) 2024-09-16T11:14:32.535 INFO:teuthology.orchestra.run.smithi162.stdout:3455 writesame 0x6b9e67f thru 0x6ba459f data_size 0x7(0x5f21 bytes) 2024-09-16T11:14:32.537 INFO:teuthology.orchestra.run.smithi162.stdout:3456 writesame 0x796e1cc thru 0x7973ab3 data_size 0x14(0x58e8 bytes) 2024-09-16T11:14:32.541 INFO:teuthology.orchestra.run.smithi162.stdout:3457 writesame 0x1e3da68 thru 0x1e40170 data_size 0x1(0x2709 bytes) 2024-09-16T11:14:32.544 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:32.545 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.04781487, 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:32.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:32.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:32.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:32.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99422 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 776192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.798030237000001, 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11283726099999999, 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99422 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:32.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742911+0000", 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045312299999999999 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742922+0000", 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1068e-05 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742939+0000", 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6685e-05 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742955+0000", 2024-09-16T11:14:32.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6419e-05 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855055+0000", 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11209968100000001 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855150+0000", 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5185000000000004e-05 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855295+0000", 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001451 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99524 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.796935340999999, 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112829479, 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.563 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99524 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744021+0000", 2024-09-16T11:14:32.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046845900000000001 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744026+0000", 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5889999999999996e-06 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744039+0000", 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3326000000000001e-05 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744045+0000", 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9460000000000003e-06 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856169+0000", 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112124052 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856233+0000", 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4344000000000006e-05 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856382+0000", 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000148763 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.568 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99537 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794422106000001, 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11344878899999999, 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.569 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99537 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746069+0000", 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746077+0000", 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1440000000000003e-06 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746081+0000", 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3540000000000002e-06 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.857981+0000", 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.111899208 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.858055+0000", 2024-09-16T11:14:32.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4374000000000003e-05 2024-09-16T11:14:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859515+0000", 2024-09-16T11:14:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014596030000000001 2024-09-16T11:14:32.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794402072, 2024-09-16T11:14:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:14:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:14:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:14:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:14:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:14:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:14:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:14:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:14:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:14:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:14:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:14:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:14:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:14:32.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:14:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:14:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:14:32.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.587 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.794132756, 2024-09-16T11:14:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:14:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:14:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.589 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:14:32.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:14:32.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:14:32.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:14:32.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:14:32.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:14:32.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:14:32.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:14:32.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:14:32.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:14:32.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:14:32.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:14:32.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:14:32.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:14:32.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.597 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:32.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.793734246, 2024-09-16T11:14:32.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:14:32.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.599 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.600 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:14:32.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.600 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:32.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:32.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:14:32.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:14:32.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:14:32.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:14:32.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:14:32.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:14:32.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:14:32.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:14:32.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:14:32.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:14:32.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:14:32.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:14:32.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:14:32.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:14:32.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.609 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:32.609 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.793445458000001, 2024-09-16T11:14:32.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:14:32.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.610 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.610 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.610 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:14:32.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.610 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:32.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:32.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:14:32.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:14:32.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:14:32.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:14:32.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:14:32.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:14:32.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:14:32.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:14:32.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:14:32.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:14:32.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99546 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.791834183000001, 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11333913599999999, 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99546 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748655+0000", 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5899999999999995e-07 2024-09-16T11:14:32.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748665+0000", 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0776e-05 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748714+0000", 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8337999999999997e-05 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748720+0000", 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1709999999999999e-06 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861601+0000", 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112880861 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861743+0000", 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014191 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861993+0000", 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025022100000000002 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.791746042, 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.622 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:32.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.626 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.791667726, 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:14:32.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:14:32.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.791633987000001, 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:14:32.631 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:32.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.635 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.791593241999999, 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.636 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:14:32.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99563 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 630784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.7884028, 2024-09-16T11:14:32.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112716318, 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99563 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7799999999999997e-07 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752087+0000", 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.314e-06 2024-09-16T11:14:32.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752091+0000", 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752093+0000", 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2170000000000001e-06 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864490+0000", 2024-09-16T11:14:32.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11239703700000001 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864596+0000", 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001063 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864801+0000", 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000205038 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788392794, 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:14:32.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:14:32.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788374737, 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:32.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.653 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.788348748000001, 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:32.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:14:32.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.787650317000001, 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.658 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:14:32.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.787637758000001, 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:14:32.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:14:32.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99591 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.783948337, 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113533827, 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99591 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756541+0000", 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5200000000000001e-06 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756545+0000", 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756555+0000", 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5070000000000006e-06 2024-09-16T11:14:32.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869736+0000", 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11318076000000001 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869888+0000", 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015207900000000001 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870073+0000", 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018570899999999999 2024-09-16T11:14:32.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:32.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:32.671 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:32.672 INFO:teuthology.orchestra.run.smithi162.stdout:3460 punch from 0x5b09009 to 0x5b172e8, (0xe2df bytes) 2024-09-16T11:14:32.673 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3461 write 0xdb9e873 thru 0xdbad1ad (0xe93b bytes) 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3463 read 0x416d606 thru 0x416f1a9 (0x1ba4 bytes) 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3465 write 0x744dec8 thru 0x7453411 (0x554a bytes) 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3467 read 0x9f0ab5f thru 0x9f0df0c (0x33ae bytes) 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3468 trunc from 0xdbad1ae to 0xd0f273b 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3470 read 0x450171d thru 0x4505fb8 (0x489c bytes) 2024-09-16T11:14:32.674 INFO:teuthology.orchestra.run.smithi162.stdout:3471 trunc from 0xd0f273b to 0x21d77ba 2024-09-16T11:14:32.675 INFO:teuthology.orchestra.run.smithi162.stdout:3472 read 0x1a3039e thru 0x1a30e11 (0xa74 bytes) 2024-09-16T11:14:32.675 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.675 INFO:teuthology.orchestra.run.smithi162.stdout:3475 write 0x375c489 thru 0x37689e7 (0xc55f bytes) 2024-09-16T11:14:32.675 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:32.667+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:32.686 INFO:teuthology.orchestra.run.smithi162.stdout:3476 write 0xa5e5434 thru 0xa5edf39 (0x8b06 bytes) 2024-09-16T11:14:32.689 INFO:teuthology.orchestra.run.smithi162.stdout:3478 read 0x93493ec thru 0x934d61f (0x4234 bytes) 2024-09-16T11:14:32.689 INFO:teuthology.orchestra.run.smithi162.stdout:3481 read 0x5e82866 thru 0x5e91791 (0xef2c bytes) 2024-09-16T11:14:32.689 INFO:teuthology.orchestra.run.smithi162.stdout:3483 punch from 0x6e50204 to 0x6e532ee, (0x30ea bytes) 2024-09-16T11:14:32.689 INFO:teuthology.orchestra.run.smithi162.stdout:3484 write 0x2d0662d thru 0x2d0f856 (0x922a bytes) 2024-09-16T11:14:32.716 INFO:teuthology.orchestra.run.smithi162.stdout:3486 writesame 0xd1522c5 thru 0xd15c16b data_size 0x1(0x9ea7 bytes) 2024-09-16T11:14:32.717 INFO:teuthology.orchestra.run.smithi162.stdout:3487 trunc from 0xd15c16c to 0xd23ac6c 2024-09-16T11:14:32.726 INFO:teuthology.orchestra.run.smithi162.stdout:3489 punch from 0x3aa1523 to 0x3aa21df, (0xcbc bytes) 2024-09-16T11:14:32.726 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.726 INFO:teuthology.orchestra.run.smithi162.stdout:3492 read 0xd146310 thru 0xd149077 (0x2d68 bytes) 2024-09-16T11:14:32.726 INFO:teuthology.orchestra.run.smithi162.stdout:3493 flatten 2024-09-16T11:14:32.737 INFO:teuthology.orchestra.run.smithi162.stdout:3499 read 0x60338bc thru 0x603ee89 (0xb5ce bytes) 2024-09-16T11:14:32.738 INFO:teuthology.orchestra.run.smithi162.stdout:3500 punch from 0x1621d4e to 0x1628458, (0x670a bytes) 2024-09-16T11:14:32.748 INFO:teuthology.orchestra.run.smithi162.stdout:3501 write 0xd8429a3 thru 0xd8463e6 (0x3a44 bytes) 2024-09-16T11:14:32.755 INFO:teuthology.orchestra.run.smithi162.stdout:3502 punch from 0xb96335a to 0xb97055e, (0xd204 bytes) 2024-09-16T11:14:32.755 INFO:teuthology.orchestra.run.smithi162.stdout:3503 punch from 0x4aa85db to 0x4aa9b39, (0x155e bytes) 2024-09-16T11:14:32.756 INFO:teuthology.orchestra.run.smithi162.stdout:3505 write 0x7bead56 thru 0x7bec4a5 (0x1750 bytes) 2024-09-16T11:14:32.757 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.757 INFO:teuthology.orchestra.run.smithi162.stdout:3507 write 0x5ebd77d thru 0x5ec4e33 (0x76b7 bytes) 2024-09-16T11:14:32.763 INFO:teuthology.orchestra.run.smithi162.stdout:3509 read 0x6c384d6 thru 0x6c3f7cf (0x72fa bytes) 2024-09-16T11:14:32.763 INFO:teuthology.orchestra.run.smithi162.stdout:3510 write 0x5c4c311 thru 0x5c4d594 (0x1284 bytes) 2024-09-16T11:14:32.765 INFO:teuthology.orchestra.run.smithi162.stdout:3512 write 0x9e881b5 thru 0x9e91c78 (0x9ac4 bytes) 2024-09-16T11:14:32.768 INFO:teuthology.orchestra.run.smithi162.stdout:3513 write 0x30c4c3f thru 0x30d171a (0xcadc bytes) 2024-09-16T11:14:32.773 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:32.771+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:32.775 INFO:teuthology.orchestra.run.smithi162.stdout:3514 writesame 0x880c7a1 thru 0x880d600 data_size 0x28(0xe60 bytes) 2024-09-16T11:14:32.789 INFO:teuthology.orchestra.run.smithi162.stdout:3515 write 0xdaba7ba thru 0xdac16fc (0x6f43 bytes) 2024-09-16T11:14:32.791 INFO:teuthology.orchestra.run.smithi162.stdout:3516 write 0x261ac97 thru 0x26286ce (0xda38 bytes) 2024-09-16T11:14:32.797 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.797 INFO:teuthology.orchestra.run.smithi162.stdout:3520 read 0x7eee1d thru 0x7f318e (0x4372 bytes) 2024-09-16T11:14:32.798 INFO:teuthology.orchestra.run.smithi162.stdout:3521 read 0xd31ec66 thru 0xd322118 (0x34b3 bytes) 2024-09-16T11:14:32.798 INFO:teuthology.orchestra.run.smithi162.stdout:3522 punch from 0x44c6a87 to 0x44d4cf6, (0xe26f bytes) 2024-09-16T11:14:32.799 INFO:teuthology.orchestra.run.smithi162.stdout:3523 punch from 0x39ca92c to 0x39d554a, (0xac1e bytes) 2024-09-16T11:14:32.799 INFO:teuthology.orchestra.run.smithi162.stdout:3524 writesame 0x7dc26fc thru 0x7dcb005 data_size 0x9(0x890a bytes) 2024-09-16T11:14:32.802 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:32.802 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:32.802 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:32.803 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:32.805 INFO:teuthology.orchestra.run.smithi162.stdout:3526 read 0x4c1eb46 thru 0x4c22328 (0x37e3 bytes) 2024-09-16T11:14:32.805 INFO:teuthology.orchestra.run.smithi162.stdout:3529 read 0xbf53ca8 thru 0xbf572a9 (0x3602 bytes) 2024-09-16T11:14:32.805 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.805 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.805 INFO:teuthology.orchestra.run.smithi162.stdout:3537 trunc from 0xdac16fd to 0xdb70810 2024-09-16T11:14:32.814 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:32.815 INFO:teuthology.orchestra.run.smithi162.stdout:3538 writesame 0x4f0363c thru 0x4f0fc3e data_size 0x3d(0xc603 bytes) 2024-09-16T11:14:32.821 INFO:teuthology.orchestra.run.smithi162.stdout:3540 punch from 0xd8ebefb to 0xd8fb3fe, (0xf503 bytes) 2024-09-16T11:14:32.821 INFO:teuthology.orchestra.run.smithi162.stdout:3541 write 0x1abe084 thru 0x1ac5d83 (0x7d00 bytes) 2024-09-16T11:14:32.823 INFO:teuthology.orchestra.run.smithi162.stdout:3543 read 0xca77d6f thru 0xca7e3bc (0x664e bytes) 2024-09-16T11:14:32.824 INFO:teuthology.orchestra.run.smithi162.stdout:3545 read 0x52f25b3 thru 0x52fe37f (0xbdcd bytes) 2024-09-16T11:14:32.825 INFO:teuthology.orchestra.run.smithi162.stdout:3546 punch from 0xcdd4020 to 0xcdd6919, (0x28f9 bytes) 2024-09-16T11:14:32.826 INFO:teuthology.orchestra.run.smithi162.stdout:3547 read 0x21ed2ed thru 0x21fcb17 (0xf82b bytes) 2024-09-16T11:14:32.826 INFO:teuthology.orchestra.run.smithi162.stdout:3548 read 0xda1dd06 thru 0xda2da9e (0xfd99 bytes) 2024-09-16T11:14:32.848 INFO:teuthology.orchestra.run.smithi162.stdout:3549 trunc from 0xdb70810 to 0x2352672 2024-09-16T11:14:32.874 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:32.869+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:32.920 INFO:teuthology.orchestra.run.smithi162.stdout:3550 write 0xe3a269c thru 0xe3a975b (0x70c0 bytes) 2024-09-16T11:14:32.928 INFO:teuthology.orchestra.run.smithi162.stdout:3551 read 0x646d1f7 thru 0x6479938 (0xc742 bytes) 2024-09-16T11:14:32.928 INFO:teuthology.orchestra.run.smithi162.stdout:3552 write 0xde5e54b thru 0xde693f3 (0xaea9 bytes) 2024-09-16T11:14:32.934 INFO:teuthology.orchestra.run.smithi162.stdout:3553 write 0xf0c343 thru 0xf12a9e (0x675c bytes) 2024-09-16T11:14:32.937 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.938 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.938 INFO:teuthology.orchestra.run.smithi162.stdout:3558 trunc from 0xe3a975c to 0xe8dfe35 2024-09-16T11:14:32.940 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:32.940 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:32.940 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:32.940 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:32.940 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:32.948 INFO:teuthology.orchestra.run.smithi162.stdout:3559 read 0xc3e9500 thru 0xc3f2e02 (0x9903 bytes) 2024-09-16T11:14:32.948 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.948 INFO:teuthology.orchestra.run.smithi162.stdout:3562 write 0xcfb52a3 thru 0xcfc0f9e (0xbcfc bytes) 2024-09-16T11:14:32.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:32.955 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:32.955 INFO:teuthology.orchestra.run.smithi162.stdout:3565 read 0x2934f6e thru 0x29371fd (0x2290 bytes) 2024-09-16T11:14:32.956 INFO:teuthology.orchestra.run.smithi162.stdout:3566 trunc from 0xe8dfe35 to 0xda05093 2024-09-16T11:14:32.980 INFO:teuthology.orchestra.run.smithi162.stdout:3567 punch from 0x23eef5d to 0x23f75ac, (0x864f bytes) 2024-09-16T11:14:32.981 INFO:teuthology.orchestra.run.smithi162.stdout:3568 write 0x38769c4 thru 0x387705f (0x69c bytes) 2024-09-16T11:14:32.981 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:32.975+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:32.988 INFO:teuthology.orchestra.run.smithi162.stdout:3573 trunc from 0xda05093 to 0xad3fdc5 2024-09-16T11:14:32.999 INFO:teuthology.orchestra.run.smithi162.stdout:3574 read 0x3f7aa71 thru 0x3f83b8f (0x911f bytes) 2024-09-16T11:14:33.010 INFO:teuthology.orchestra.run.smithi162.stdout:3577 write 0xb8b0d16 thru 0xb8bdbbe (0xcea9 bytes) 2024-09-16T11:14:33.013 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.029 INFO:teuthology.orchestra.run.smithi162.stdout:3580 trunc from 0xb8bdbbf to 0x1be4c7e 2024-09-16T11:14:33.075 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:33.071+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:33.078 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:33.078 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:33.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:33.078 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:33.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996905196, 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:14:33.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996890737999999, 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:33.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:14:33.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:14:33.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996873537999999, 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:14:33.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:14:33.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996860018, 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:33.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:14:33.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.099 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996851476, 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:33.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.104 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996841332000001, 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:14:33.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996827825, 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:14:33.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.114 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996814233, 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:33.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.119 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996789951, 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:33.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.99675579, 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:14:33.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:14:33.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996722393000001, 2024-09-16T11:14:33.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:14:33.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:14:33.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996699747999999, 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:14:33.134 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:14:33.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996684173, 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.139 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.996667138999999, 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.144 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:33.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:14:33.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.993034175, 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.149 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:14:33.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:14:33.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.992999833000001, 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:14:33.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:14:33.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:14:33.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.992983886999999, 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.992968275999999, 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:14:33.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.992953188, 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:14:33.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:14:33.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.992938386000001, 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:33.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:14:33.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:14:33.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:14:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:14:33.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.179 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.179 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.179 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:33.180 INFO:teuthology.orchestra.run.smithi162.stdout:3581 write 0xbd59162 thru 0xbd6024e (0x70ed bytes) 2024-09-16T11:14:33.180 INFO:teuthology.orchestra.run.smithi162.stdout:3582 write 0x7f38c84 thru 0x7f48b73 (0xfef0 bytes) 2024-09-16T11:14:33.181 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3585 write 0xa9738fa thru 0xa97a8ff (0x7006 bytes) 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3587 writesame 0xb86917f thru 0xb86ae7d data_size 0x1(0x1cff bytes) 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3588 write 0x972b4d0 thru 0x9734361 (0x8e92 bytes) 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3590 read 0x821898a thru 0x8221ec4 (0x953b bytes) 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3592 read 0x7da2e35 thru 0x7da7ebd (0x5089 bytes) 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3593 write 0x6d6b69c thru 0x6d6c4cf (0xe34 bytes) 2024-09-16T11:14:33.182 INFO:teuthology.orchestra.run.smithi162.stdout:3594 trunc from 0xbd6024f to 0x4d33ce 2024-09-16T11:14:33.183 INFO:teuthology.orchestra.run.smithi162.stdout:3596 read 0x2936d2 thru 0x2a3286 (0xfbb5 bytes) 2024-09-16T11:14:33.183 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:33.173+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:33.215 INFO:teuthology.orchestra.run.smithi162.stdout:3598 writesame 0xd87c00b thru 0xd87d390 data_size 0x31(0x1386 bytes) 2024-09-16T11:14:33.215 INFO:teuthology.orchestra.run.smithi162.stdout:3600 writesame 0x3a623c0 thru 0x3a65901 data_size 0x2(0x3542 bytes) 2024-09-16T11:14:33.222 INFO:teuthology.orchestra.run.smithi162.stdout:3602 write 0xc7bfa3b thru 0xc7c381d (0x3de3 bytes) 2024-09-16T11:14:33.229 INFO:teuthology.orchestra.run.smithi162.stdout:3604 write 0xe6583e thru 0xe65cf1 (0x4b4 bytes) 2024-09-16T11:14:33.235 INFO:teuthology.orchestra.run.smithi162.stdout:3605 read 0x5b83699 thru 0x5b8a39c (0x6d04 bytes) 2024-09-16T11:14:33.235 INFO:teuthology.orchestra.run.smithi162.stdout:3606 read 0xabfbfb6 thru 0xac076b0 (0xb6fb bytes) 2024-09-16T11:14:33.235 INFO:teuthology.orchestra.run.smithi162.stdout:3607 read 0xa63a5c5 thru 0xa63cf82 (0x29be bytes) 2024-09-16T11:14:33.255 INFO:teuthology.orchestra.run.smithi162.stdout:3608 trunc from 0xd87d391 to 0x3d8d2cb 2024-09-16T11:14:33.268 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.268 INFO:teuthology.orchestra.run.smithi162.stdout:3612 punch from 0x3295ee8 to 0x329e50b, (0x8623 bytes) 2024-09-16T11:14:33.268 INFO:teuthology.orchestra.run.smithi162.stdout:3613 read 0x170984c thru 0x1713cb4 (0xa469 bytes) 2024-09-16T11:14:33.268 INFO:teuthology.orchestra.run.smithi162.stdout:3615 write 0xd3a55b thru 0xd3c340 (0x1de6 bytes) 2024-09-16T11:14:33.270 INFO:teuthology.orchestra.run.smithi162.stdout:3616 write 0x2475453 thru 0x2481863 (0xc411 bytes) 2024-09-16T11:14:33.275 INFO:teuthology.orchestra.run.smithi162.stdout:3617 write 0x3a4566 thru 0x3a8186 (0x3c21 bytes) 2024-09-16T11:14:33.278 INFO:teuthology.orchestra.run.smithi162.stdout:3618 read 0x53afd3 thru 0x53d381 (0x23af bytes) 2024-09-16T11:14:33.281 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:33.275+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:33.282 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:33.287 INFO:teuthology.orchestra.run.smithi162.stdout:3620 trunc from 0x3d8d2cb to 0x8518108 2024-09-16T11:14:33.298 INFO:teuthology.orchestra.run.smithi162.stdout:3621 trunc from 0x8518108 to 0x71fd737 2024-09-16T11:14:33.309 INFO:teuthology.orchestra.run.smithi162.stdout:3622 writesame 0x66e73b8 thru 0x66f186f data_size 0x38(0xa4b8 bytes) 2024-09-16T11:14:33.315 INFO:teuthology.orchestra.run.smithi162.stdout:3624 writesame 0x636d711 thru 0x636dcab data_size 0x7(0x59b bytes) 2024-09-16T11:14:33.323 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.328 INFO:teuthology.orchestra.run.smithi162.stdout:3627 trunc from 0x71fd737 to 0x4a6e79a 2024-09-16T11:14:33.330 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "pg_update_log_missing(4.f epoch 90/31 rep_tid 1678 entries 90'301 (0'0) error 4:f4805d1b:::rbd_data.11b5eb152d9.0000000000000009:head by client.4533.0:133175 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 90'299)", 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.328743+0000", 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00054549199999999996, 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00058432099999999995, 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.328743+0000", 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.328743+0000", 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.328744+0000", 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2559999999999999e-06 2024-09-16T11:14:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.328761+0000", 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7388e-05 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.328795+0000", 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3528999999999997e-05 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.328808+0000", 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2843999999999999e-05 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.329041+0000", 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023317300000000001 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:14:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:14:33.335 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:33.341 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:33.342 INFO:teuthology.orchestra.run.smithi162.stdout:3628 read 0x485a547 thru 0x486192b (0x73e5 bytes) 2024-09-16T11:14:33.342 INFO:teuthology.orchestra.run.smithi162.stdout:3629 punch from 0x4258e0f to 0x425c9fa, (0x3beb bytes) 2024-09-16T11:14:33.359 INFO:teuthology.orchestra.run.smithi162.stdout:3630 write 0xa59a4d8 thru 0xa59b99a (0x14c3 bytes) 2024-09-16T11:14:33.381 INFO:teuthology.orchestra.run.smithi162.stdout:3632 write 0xe949343 thru 0xe94a3b7 (0x1075 bytes) 2024-09-16T11:14:33.381 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:33.379+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:33.388 INFO:teuthology.orchestra.run.smithi162.stdout:3634 punch from 0x6c5ea01 to 0x6c6757a, (0x8b79 bytes) 2024-09-16T11:14:33.388 INFO:teuthology.orchestra.run.smithi162.stdout:3635 read 0xbb8a5cd thru 0xbb9a0c5 (0xfaf9 bytes) 2024-09-16T11:14:33.388 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.389 INFO:teuthology.orchestra.run.smithi162.stdout:3637 read 0x700d1fc thru 0x700eafd (0x1902 bytes) 2024-09-16T11:14:33.389 INFO:teuthology.orchestra.run.smithi162.stdout:3638 writesame 0x5fb709b thru 0x5fbec58 data_size 0x2f(0x7bbe bytes) 2024-09-16T11:14:33.391 INFO:teuthology.orchestra.run.smithi162.stdout:3639 read 0x52a361a thru 0x52acd1d (0x9704 bytes) 2024-09-16T11:14:33.391 INFO:teuthology.orchestra.run.smithi162.stdout:3640 punch from 0x40682ab to 0x406b0cb, (0x2e20 bytes) 2024-09-16T11:14:33.391 INFO:teuthology.orchestra.run.smithi162.stdout:3642 read 0x20ce203 thru 0x20d65ca (0x83c8 bytes) 2024-09-16T11:14:33.391 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.391 INFO:teuthology.orchestra.run.smithi162.stdout:3644 read 0x52c736f thru 0x52d05bf (0x9251 bytes) 2024-09-16T11:14:33.391 INFO:teuthology.orchestra.run.smithi162.stdout:3645 write 0x5952908 thru 0x5962241 (0xf93a bytes) 2024-09-16T11:14:33.393 INFO:teuthology.orchestra.run.smithi162.stdout:3647 read 0x61cbfba thru 0x61cde6b (0x1eb2 bytes) 2024-09-16T11:14:33.394 INFO:teuthology.orchestra.run.smithi162.stdout:3648 writesame 0xcd98af3 thru 0xcda26da data_size 0xc(0x9be8 bytes) 2024-09-16T11:14:33.400 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.400 INFO:teuthology.orchestra.run.smithi162.stdout:3651 write 0x28976e6 thru 0x28a412a (0xca45 bytes) 2024-09-16T11:14:33.402 INFO:teuthology.orchestra.run.smithi162.stdout:3652 read 0x51e7586 thru 0x51f4aa4 (0xd51f bytes) 2024-09-16T11:14:33.403 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:33.403 INFO:teuthology.orchestra.run.smithi162.stdout:3654 write 0x67670f thru 0x681409 (0xacfb bytes) 2024-09-16T11:14:33.406 INFO:teuthology.orchestra.run.smithi162.stdout:3655 punch from 0x64814af to 0x6485a22, (0x4573 bytes) 2024-09-16T11:14:33.406 INFO:teuthology.orchestra.run.smithi162.stdout:3656 read 0x150216f thru 0x150e5fa (0xc48c bytes) 2024-09-16T11:14:33.406 INFO:teuthology.orchestra.run.smithi162.stdout:3659 write 0x8d4b464 thru 0x8d57516 (0xc0b3 bytes) 2024-09-16T11:14:33.408 INFO:teuthology.orchestra.run.smithi162.stdout:3660 write 0xa09923c thru 0xa09c321 (0x30e6 bytes) 2024-09-16T11:14:33.412 INFO:teuthology.orchestra.run.smithi162.stdout:3662 read 0x8af78cc thru 0x8aff2cd (0x7a02 bytes) 2024-09-16T11:14:33.412 INFO:teuthology.orchestra.run.smithi162.stdout:3663 writesame 0xaed165e thru 0xaee0d12 data_size 0x1(0xf6b5 bytes) 2024-09-16T11:14:33.415 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:33.452 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:33.454 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:33.454 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:33.454 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:33.455 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:33.455 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:33.460 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:14:33.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:33.469 INFO:teuthology.orchestra.run.smithi162.stdout:3664 read 0x6fe41de thru 0x6ff35f9 (0xf41c bytes) 2024-09-16T11:14:33.470 INFO:teuthology.orchestra.run.smithi162.stdout:3665 writesame 0xc9ccf79 thru 0xc9d93c1 data_size 0x1(0xc449 bytes) 2024-09-16T11:14:33.477 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:33.482 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:33.479+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:33.496 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:33.513 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:33.533 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:33.557 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:33.565 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:33.566 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:33.575 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:33.580 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:33.582 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:33.583 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:33.584 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:33.579+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:33.684 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:33.684 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:33.684 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:33.679+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:33.693 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:33.785 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:33.777+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:33.794 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:33.794 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:33.803 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:33.886 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:33.883+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:33.902 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:33.902 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:33.911 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:33.986 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:33.981+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:34.010 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.010 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.019 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:34.087 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:34.083+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:34.163 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.164 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.172 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:34.188 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:34.183+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:34.273 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.273 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.282 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:34.288 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:34.283+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:34.381 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.381 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.389 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:34.387+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:34.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:34.449 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:14:34.491 INFO:teuthology.orchestra.run.smithi162.stdout:3666 trunc from 0xe94a3b8 to 0x9c84094 2024-09-16T11:14:34.492 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:34.487+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:34.500 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.500 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.511 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:34.532 INFO:teuthology.orchestra.run.smithi162.stdout:3667 writesame 0xdbe6564 thru 0xdbea545 data_size 0x25(0x3fe2 bytes) 2024-09-16T11:14:34.532 INFO:teuthology.orchestra.run.smithi162.stdout:3670 write 0xa0a84a2 thru 0xa0b2c82 (0xa7e1 bytes) 2024-09-16T11:14:34.535 INFO:teuthology.orchestra.run.smithi162.stdout:3671 write 0xa98d631 thru 0xa997c3c (0xa60c bytes) 2024-09-16T11:14:34.538 INFO:teuthology.orchestra.run.smithi162.stdout:3673 write 0xa6ebce7 thru 0xa6f58e9 (0x9c03 bytes) 2024-09-16T11:14:34.545 INFO:teuthology.orchestra.run.smithi162.stdout:3674 trunc from 0xdbea546 to 0xbe5d57e 2024-09-16T11:14:34.564 INFO:teuthology.orchestra.run.smithi162.stdout:3676 trunc from 0xbe5d57e to 0xe63bba1 2024-09-16T11:14:34.590 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:34.585+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:34.598 INFO:teuthology.orchestra.run.smithi162.stdout:3677 trunc from 0xe63bba1 to 0x1761cfb 2024-09-16T11:14:34.608 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.608 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:34.628 INFO:teuthology.orchestra.run.smithi162.stdout:3678 read 0x12ad074 thru 0x12ba365 (0xd2f2 bytes) 2024-09-16T11:14:34.628 INFO:teuthology.orchestra.run.smithi162.stdout:3680 punch from 0x7ffd6 to 0x872cf, (0x72f9 bytes) 2024-09-16T11:14:34.654 INFO:teuthology.orchestra.run.smithi162.stdout:3681 write 0xa5b2ce1 thru 0xa5b7f04 (0x5224 bytes) 2024-09-16T11:14:34.657 INFO:teuthology.orchestra.run.smithi162.stdout:3682 punch from 0x353391e to 0x3541a09, (0xe0eb bytes) 2024-09-16T11:14:34.675 INFO:teuthology.orchestra.run.smithi162.stdout:3683 write 0xdaacdd2 thru 0xdaafb2d (0x2d5c bytes) 2024-09-16T11:14:34.682 INFO:teuthology.orchestra.run.smithi162.stdout:3684 write 0x4ee8cad thru 0x4eeb3a0 (0x26f4 bytes) 2024-09-16T11:14:34.689 INFO:teuthology.orchestra.run.smithi162.stdout:3686 punch from 0x14304f to 0x14434d, (0x12fe bytes) 2024-09-16T11:14:34.689 INFO:teuthology.orchestra.run.smithi162.stdout:3687 clone 30 order 23 su 4194304 sc 15 2024-09-16T11:14:34.691 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:34.687+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:34.717 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.717 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.726 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:34.792 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:34.785+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:34.820 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.820 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.829 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:34.893 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:34.891+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:34.929 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:34.929 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:34.938 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:34.994 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:34.991+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:35.047 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:35.048 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:35.058 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:35.095 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:35.091+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:35.161 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:35.161 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:35.166 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 6, 7, 2] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:35.166 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:35.166 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T11:14:35.166 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:14:35.166 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:14:35.166 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-16T11:14:35.170 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:35.195 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:35.191+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:35.267 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:35.268 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:35.274 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:35.274 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T11:14:35.274 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:35.276 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:35.296 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:35.289+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:35.397 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:35.389+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:35.423 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:35.423 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:35.432 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:35.498 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:35.495+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:35.543 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:35.543 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:35.553 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:35.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:35.601 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:35.599+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:35.701 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:35.697+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:35.705 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:35.705 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:35.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:35.732 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:35.754 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:35.770 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-16T11:14:35.781 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:35.802 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:35.797+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:35.813 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:35.831 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:35.831 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:35.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.831 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134132 4.1c 4:3f6dfe25:::rbd_data.11b5cb71539e.0000000000000004:1e [read 1146880~32768 out=32768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.831 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.827934+0000", 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00053313799999999995, 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000537486, 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134132 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827934+0000", 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827934+0000", 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827936+0000", 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.17e-06 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827941+0000", 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4229999999999996e-06 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827954+0000", 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3061000000000001e-05 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827960+0000", 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3029999999999996e-06 2024-09-16T11:14:35.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828313+0000", 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035216199999999999 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828385+0000", 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2520000000000004e-05 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:35.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828472+0000", 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6847000000000004e-05 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134133 4.4 4.656e1224 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.827967+0000", 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00050036999999999998, 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001208055, 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.836 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134133 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827967+0000", 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827967+0000", 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827968+0000", 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.46e-07 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827970+0000", 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5550000000000001e-06 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827976+0000", 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.879e-06 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.827980+0000", 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0249999999999996e-06 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134141 4.1c 4.a47fb6fc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828007+0000", 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00046063099999999997, 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0012243449999999999, 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134141 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828007+0000", 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828007+0000", 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828007+0000", 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4099999999999999e-07 2024-09-16T11:14:35.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828010+0000", 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4090000000000001e-06 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828016+0000", 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5549999999999999e-06 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828020+0000", 2024-09-16T11:14:35.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7850000000000002e-06 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134142 4.4 4.656e1224 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828025+0000", 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00044305000000000001, 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00126092, 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134142 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828025+0000", 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828025+0000", 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828025+0000", 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4799999999999999e-07 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828027+0000", 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4329999999999998e-06 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828033+0000", 2024-09-16T11:14:35.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8259999999999998e-06 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828037+0000", 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7359999999999999e-06 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134147 4.1c 4.a47fb6fc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828041+0000", 2024-09-16T11:14:35.846 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.000426613, 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001299995, 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134147 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828041+0000", 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828041+0000", 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828041+0000", 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3799999999999998e-07 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828044+0000", 2024-09-16T11:14:35.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3630000000000001e-06 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828049+0000", 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6710000000000004e-06 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828053+0000", 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5470000000000001e-06 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134150 4.1c 4.a47fb6fc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828056+0000", 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00041129099999999999, 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001337782, 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134150 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828056+0000", 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828056+0000", 2024-09-16T11:14:35.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828057+0000", 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.72e-07 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828059+0000", 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1550000000000001e-06 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828065+0000", 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.694e-06 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828068+0000", 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4450000000000001e-06 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134151 4.1c 4.a47fb6fc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.853 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828499+0000", 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4294967295.9999685, 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00095014199999999996, 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134151 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828499+0000", 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828499+0000", 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828500+0000", 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5099999999999998e-07 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828504+0000", 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1409999999999998e-06 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828514+0000", 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0604e-05 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828522+0000", 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7100000000000007e-06 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134155 4.1c 4.a47fb6fc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828529+0000", 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4294967295.9999385, 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00097457200000000005, 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134155 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.857 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828529+0000", 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828529+0000", 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828530+0000", 2024-09-16T11:14:35.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.7000000000000003e-07 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828533+0000", 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.58e-06 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828541+0000", 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5839999999999997e-06 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828546+0000", 2024-09-16T11:14:35.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2390000000000002e-06 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134162 4.1c 4.a47fb6fc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828702+0000", 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4294967295.9997654, 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00085499600000000001, 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134162 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828702+0000", 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828702+0000", 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828703+0000", 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9400000000000005e-07 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828706+0000", 2024-09-16T11:14:35.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8310000000000001e-06 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828713+0000", 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2779999999999996e-06 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828717+0000", 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7589999999999999e-06 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134163 4.4 4.656e1224 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828721+0000", 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4294967295.9997468, 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000893536, 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134163 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828721+0000", 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828721+0000", 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828722+0000", 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4600000000000004e-07 2024-09-16T11:14:35.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828724+0000", 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4099999999999998e-06 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828731+0000", 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7240000000000003e-06 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828736+0000", 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2279999999999998e-06 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134164 4.4 4.656e1224 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828740+0000", 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4294967295.9997277, 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000928585, 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134164 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828740+0000", 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828740+0000", 2024-09-16T11:14:35.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828740+0000", 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6800000000000001e-07 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828743+0000", 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3190000000000002e-06 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828750+0000", 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1069999999999999e-06 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828754+0000", 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9609999999999999e-06 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.870 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:134165 4.4 4.656e1224 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:35.828758+0000", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4294967295.9997096, 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00096417200000000001, 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "queued for pg", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 134165 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828758+0000", 2024-09-16T11:14:35.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828758+0000", 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828758+0000", 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4299999999999998e-07 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:35.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828760+0000", 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3180000000000001e-06 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828767+0000", 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5509999999999996e-06 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:35.828771+0000", 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0500000000000002e-06 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:35.874 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.874 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:35.874 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:14:35.874 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 12 2024-09-16T11:14:35.874 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:35.875 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:35.875 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:35.881 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:35.902 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:35.897+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:35.915 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:35.935 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:35.937 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:35.937 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:35.937 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:35.937 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:35.937 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:35.937 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:35.938 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:35.938 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:35.974 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:35.974 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:35.974 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:35.974 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:35.974 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:35.982 INFO:teuthology.orchestra.run.smithi162.stdout:3690 write 0xc98500d thru 0xc98e8f8 (0x98ec bytes) 2024-09-16T11:14:35.984 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:35.994 INFO:teuthology.orchestra.run.smithi162.stdout:3692 write 0xc3975cb thru 0xc39d66c (0x60a2 bytes) 2024-09-16T11:14:35.999 INFO:teuthology.orchestra.run.smithi162.stdout:3693 read 0x8a4122 thru 0x8ae838 (0xa717 bytes) 2024-09-16T11:14:36.000 INFO:teuthology.orchestra.run.smithi162.stdout:3695 read 0x5422c97 thru 0x54237de (0xb48 bytes) 2024-09-16T11:14:36.000 INFO:teuthology.orchestra.run.smithi162.stdout:3697 writesame 0x17b6b28 thru 0x17b9f11 data_size 0xf(0x33ea bytes) 2024-09-16T11:14:36.002 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:35.999+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:36.009 INFO:teuthology.orchestra.run.smithi162.stdout:3698 punch from 0x24c44ee to 0x24cc2f6, (0x7e08 bytes) 2024-09-16T11:14:36.010 INFO:teuthology.orchestra.run.smithi162.stdout:3699 punch from 0x28c06ad to 0x28c8a9f, (0x83f2 bytes) 2024-09-16T11:14:36.010 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:36.010 INFO:teuthology.orchestra.run.smithi162.stdout:3702 writesame 0x742d35d thru 0x743c455 data_size 0x1(0xf0f9 bytes) 2024-09-16T11:14:36.052 INFO:teuthology.orchestra.run.smithi162.stdout:3703 read 0x3c81a9 thru 0x3cc210 (0x4068 bytes) 2024-09-16T11:14:36.053 INFO:teuthology.orchestra.run.smithi162.stdout:3704 clone 31 order 18 su 4096 sc 10 2024-09-16T11:14:36.093 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012734042, 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:14:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012719583999999, 2024-09-16T11:14:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:14:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:14:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:14:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012702384000001, 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:14:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012688863999999, 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:14:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012680322, 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:14:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012670178, 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:14:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:14:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012656671, 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012643079, 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:14:36.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:14:36.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:14:36.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012618797, 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:14:36.136 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:14:36.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012584636, 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:14:36.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:14:36.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.146 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012551239, 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.151 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012528594000001, 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:14:36.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:14:36.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:14:36.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:14:36.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.157 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012513019, 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.158 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:14:36.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.162 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.012495984999999, 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:14:36.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.008863021, 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:14:36.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:14:36.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.008828679000001, 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:14:36.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:36.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:14:36.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.008812732999999, 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:14:36.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:14:36.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.008797122000001, 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:14:36.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.008782033999999, 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:14:36.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.008767232, 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:14:36.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.198 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.198 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:36.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:36.200 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:36.193+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:36.300 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:36.297+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:36.349 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:36.349 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:36.349 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:36.359 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:36.400 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:36.397+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:36.455 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:36.455 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:36.455 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:36.455 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:36.456 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:36.461 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:14:36.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:36.501 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:36.493+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:36.566 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone30 from 0xdaafb2e (overlap 0xdaafb2e) to 0x8336382 2024-09-16T11:14:36.602 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:36.597+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:36.603 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:36.603 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:36.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:36.603 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:36.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49776 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1351680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.466200897999997, 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050449183000000002, 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49776 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.604 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133180+0000", 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.36e-07 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133181+0000", 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.305e-06 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133184+0000", 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9119999999999998e-06 2024-09-16T11:14:36.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133186+0000", 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0810000000000001e-06 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183293+0000", 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050106792999999997 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183376+0000", 2024-09-16T11:14:36.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2792000000000003e-05 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183629+0000", 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000253064 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.608 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.466067154999998, 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:36.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.466048766999997, 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.613 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:36.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.617 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.465748009999999, 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.618 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:36.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:36.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.465718305000003, 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.624 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:36.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.465704907000003, 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.628 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:36.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.632 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.465691102999997, 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:36.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.465316104000003, 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:36.637 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.638 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:36.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:36.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.451772073000001, 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:36.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.451750498999999, 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.648 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:36.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.451734000000002, 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:36.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.451717924, 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:36.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.451334350000003, 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.661 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:36.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.439252173, 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:36.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.439243972, 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:36.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:36.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.439235949999997, 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:36.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:36.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 19.755757071000001, 2024-09-16T11:14:36.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:36.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:14:36.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:14:36.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:36.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.52233872, 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:14:36.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:36.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.1359453469999998, 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:36.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:14:36.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:14:36.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.1843406989999998, 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:36.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:14:36.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:14:36.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:36.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:36.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:36.703 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:36.704 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:36.705 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:36.697+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:36.737 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-16T11:14:36.805 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:36.799+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:36.809 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:36.809 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:36.809 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:36.809 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:36.819 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:36.906 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:36.903+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:36.915 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:36.915 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:36.915 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:36.915 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:36.915 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:36.924 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:37.007 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:37.003+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:37.025 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947779586999999, 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.031 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947760220999999, 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:37.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:14:37.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947719459, 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.037 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:37.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:37.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.041 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947688661000001, 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:37.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:37.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:37.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:14:37.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947670072999999, 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:37.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:14:37.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947652021, 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.052 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:37.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.057 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947632093999999, 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:37.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:14:37.062 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947613255, 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:37.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:37.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:14:37.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947421306000001, 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:37.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:37.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.947402026000001, 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:37.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943931761, 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:37.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943910458, 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:14:37.083 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:37.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943900386999999, 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:37.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:14:37.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943872084000001, 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.094 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:37.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943858155000001, 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.099 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:37.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:37.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.104 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943849392000001, 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:37.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.109 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943825573, 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:14:37.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:14:37.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943794451, 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:14:37.115 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:14:37.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943746711999999, 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:14:37.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 14.943724604, 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.126 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:14:37.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.130 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:37.131 INFO:teuthology.orchestra.run.smithi162.stdout:3705 punch from 0x9c55901 to 0x9c611df, (0xb8de bytes) 2024-09-16T11:14:37.132 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:37.132 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:37.129+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:37.134 INFO:teuthology.orchestra.run.smithi162.stdout:3706 read 0xa93f12 thru 0xa96e84 (0x2f73 bytes) 2024-09-16T11:14:37.138 INFO:teuthology.orchestra.run.smithi162.stdout:3707 punch from 0xe7006 to 0xf4316, (0xd310 bytes) 2024-09-16T11:14:37.139 INFO:teuthology.orchestra.run.smithi162.stdout:3708 writesame 0xce4b2ed thru 0xce560ac data_size 0x40(0xadc0 bytes) 2024-09-16T11:14:37.162 INFO:teuthology.orchestra.run.smithi162.stdout:3709 write 0x37b7247 thru 0x37c105c (0x9e16 bytes) 2024-09-16T11:14:37.232 INFO:teuthology.orchestra.run.smithi162.stdout:3710 read 0x8e24f thru 0x90296 (0x2048 bytes) 2024-09-16T11:14:37.233 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:37.225+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:37.233 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.233 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:37.233 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:37.233 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:37.237 INFO:teuthology.orchestra.run.smithi162.stdout:3711 punch from 0x4cce47 to 0x4d8ff8, (0xc1b1 bytes) 2024-09-16T11:14:37.237 INFO:teuthology.orchestra.run.smithi162.stdout:3712 punch from 0x3052d4d to 0x3058ae3, (0x5d96 bytes) 2024-09-16T11:14:37.237 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:37.237 INFO:teuthology.orchestra.run.smithi162.stdout:3715 write 0x1cd1d73 thru 0x1cdaac8 (0x8d56 bytes) 2024-09-16T11:14:37.242 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:37.269 INFO:teuthology.orchestra.run.smithi162.stdout:3716 punch from 0x3aec18c to 0x3af8351, (0xc1c5 bytes) 2024-09-16T11:14:37.269 INFO:teuthology.orchestra.run.smithi162.stdout:3718 writesame 0x9a50d27 thru 0x9a5b70a data_size 0x53(0xa9e4 bytes) 2024-09-16T11:14:37.285 INFO:teuthology.orchestra.run.smithi162.stdout:3719 trunc from 0xdaafb2e to 0xde8b828 2024-09-16T11:14:37.333 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:37.325+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:37.333 INFO:teuthology.orchestra.run.smithi162.stdout:3720 read 0x58db68b thru 0x58e1162 (0x5ad8 bytes) 2024-09-16T11:14:37.336 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.336 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:37.336 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:37.336 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:37.336 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:37.345 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:37.355 INFO:teuthology.orchestra.run.smithi162.stdout:3721 trunc from 0xde8b828 to 0x3491af0 2024-09-16T11:14:37.434 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:37.429+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.576195428999995, 2024-09-16T11:14:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.40822378, 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:37.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408212218999999, 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.451 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:37.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408199017000001, 2024-09-16T11:14:37.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:37.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408186577000002, 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.460 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408173066000003, 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:37.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:37.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:37.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408162150999999, 2024-09-16T11:14:37.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:37.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.472 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408136233, 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.473 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:37.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408120779000001, 2024-09-16T11:14:37.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408110893, 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:37.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:37.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408101129999999, 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:37.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408090952999999, 2024-09-16T11:14:37.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.494 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408074470999999, 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:37.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:37.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408059774000002, 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.499 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:37.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.503 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408041331, 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:37.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:37.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408024382000001, 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.508 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:37.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:37.512 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.408005168999999, 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.407987953000003, 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:37.517 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:37.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:37.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.407966698999999, 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.522 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:37.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:37.526 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.407946731000003, 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:37.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.530 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.531 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:37.531 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:14:37.532 INFO:teuthology.orchestra.run.smithi162.stdout:3722 trunc from 0x3491af0 to 0x8d90990 2024-09-16T11:14:37.533 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:37.534 INFO:teuthology.orchestra.run.smithi162.stdout:3723 write 0x577df23 thru 0x577eefc (0xfda bytes) 2024-09-16T11:14:37.534 INFO:teuthology.orchestra.run.smithi162.stdout:3724 read 0x45124c3 thru 0x451f088 (0xcbc6 bytes) 2024-09-16T11:14:37.534 INFO:teuthology.orchestra.run.smithi162.stdout:3725 read 0x3d074fc thru 0x3d08eb5 (0x19ba bytes) 2024-09-16T11:14:37.534 INFO:teuthology.orchestra.run.smithi162.stdout:3726 write 0x17a7e42 thru 0x17ab822 (0x39e1 bytes) 2024-09-16T11:14:37.535 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:37.529+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:37.564 INFO:teuthology.orchestra.run.smithi162.stdout:3727 punch from 0xc68185 to 0xc68667, (0x4e2 bytes) 2024-09-16T11:14:37.565 INFO:teuthology.orchestra.run.smithi162.stdout:3729 punch from 0x19ad74 to 0x1a6b1e, (0xbdaa bytes) 2024-09-16T11:14:37.565 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:37.570 INFO:teuthology.orchestra.run.smithi162.stdout:3732 trunc from 0x8d90990 to 0x544e2c8 2024-09-16T11:14:37.610 INFO:teuthology.orchestra.run.smithi162.stdout:3734 write 0xdc960e5 thru 0xdc9f409 (0x9325 bytes) 2024-09-16T11:14:37.631 INFO:teuthology.orchestra.run.smithi162.stdout:3736 write 0x507d204 thru 0x508cde7 (0xfbe4 bytes) 2024-09-16T11:14:37.636 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:37.631+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:37.648 INFO:teuthology.orchestra.run.smithi162.stdout:3738 read 0x9f807e8 thru 0x9f8bf5d (0xb776 bytes) 2024-09-16T11:14:37.651 INFO:teuthology.orchestra.run.smithi162.stdout:3739 trunc from 0xdc9f40a to 0xc6b1719 2024-09-16T11:14:37.652 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.652 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:37.652 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:37.652 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:37.662 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:37.681 INFO:teuthology.orchestra.run.smithi162.stdout:3742 write 0xe874b61 thru 0xe88353c (0xe9dc bytes) 2024-09-16T11:14:37.712 INFO:teuthology.orchestra.run.smithi162.stdout:3743 read 0x243a39b thru 0x243c49e (0x2104 bytes) 2024-09-16T11:14:37.712 INFO:teuthology.orchestra.run.smithi162.stdout:3744 read 0x698c17a thru 0x6995252 (0x90d9 bytes) 2024-09-16T11:14:37.713 INFO:teuthology.orchestra.run.smithi162.stdout:3745 read 0xdc90209 thru 0xdc911d1 (0xfc9 bytes) 2024-09-16T11:14:37.713 INFO:teuthology.orchestra.run.smithi162.stdout:3746 write 0xb9fbe47 thru 0xb9ff4a7 (0x3661 bytes) 2024-09-16T11:14:37.737 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:37.729+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:37.737 INFO:teuthology.orchestra.run.smithi162.stdout:3748 trunc from 0xe88353d to 0xa39978f 2024-09-16T11:14:37.756 INFO:teuthology.orchestra.run.smithi162.stdout:3749 writesame 0x29a2454 thru 0x29ab43f data_size 0x7a(0x8fec bytes) 2024-09-16T11:14:37.769 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:37.769 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:37.777 INFO:teuthology.orchestra.run.smithi162.stdout:3750 clone 32 order 22 su 1048576 sc 5 2024-09-16T11:14:37.778 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:37.838 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:37.835+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:37.929 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:37.929 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:37.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:37.929 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:37.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.929 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.740173433000002, 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:37.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.735814906000002, 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.935 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:37.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:37.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.732176375999998, 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.940 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.73215759, 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:37.944 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:37.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:37.948 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.732140266999998, 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.949 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.953 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.732122797999999, 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728282339000003, 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:37.958 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.962 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728272224000001, 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:37.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.967 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728262342999997, 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:37.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:37.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728253166999998, 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.972 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:37.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:37.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728237100000001, 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:37.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:37.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728220704999998, 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:37.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.985 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728211131000002, 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:37.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.728201595000002, 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:37.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.993 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.724144938000002, 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:37.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:37.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.724135556, 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:37.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:37.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:38.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:38.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.724126898000002, 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:38.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:38.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:38.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.724117663999998, 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:38.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:38.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 21.030364712000001, 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:38.010 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:14:38.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:38.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:14:38.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.3445294580000002, 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:38.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:14:38.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:14:38.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:14:38.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:38.022 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:38.022 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:38.022 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:38.024 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:38.024 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:38.017+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:38.124 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:38.119+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:38.129 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:38.129 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:38.129 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:38.129 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:38.138 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:38.161 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:38.182 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:38.203 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:38.223 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:38.224 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:38.217+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:38.239 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:38.239 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:38.239 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:38.245 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:38.249 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:38.268 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:38.289 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:38.310 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:38.313 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:38.313 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:38.313 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:38.313 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:38.313 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:38.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:38.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:38.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:38.325 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:38.317+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:38.346 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:38.346 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:38.355 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:38.427 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:38.425+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:38.459 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:38.460 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:38.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:38.529 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:38.527+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:38.571 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:38.571 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:38.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:38.604 INFO:teuthology.orchestra.run.smithi162.stdout:flattening image image_client.0-clone31 2024-09-16T11:14:38.630 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:38.627+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:38.691 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:38.691 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:38.701 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:38.745 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:38.743+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:38.832 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:38.825+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:38.869 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:38.869 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:38.879 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:38.932 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:38.925+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:38.987 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:38.987 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:38.999 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:39.033 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:39.031+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:39.113 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.113 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.124 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:39.134 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:39.129+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:39.234 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:39.229+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:39.241 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.241 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.253 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:39.335 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:39.329+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:39.360 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.360 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.370 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:39.436 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:39.431+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:39.496 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.496 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.508 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:39.534 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:14:39.537 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:39.529+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:39.605 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.605 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:39.638 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:39.635+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:39.731 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.731 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.739 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:39.735+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:39.741 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:39.744 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-16T11:14:39.840 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:39.833+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:39.857 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.857 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.866 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:39.941 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:39.939+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:39.972 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:39.972 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:39.981 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:40.023 INFO:teuthology.orchestra.run.smithi162.stdout:3752 read 0xcc080c thru 0xcc7b58 (0x734d bytes) 2024-09-16T11:14:40.042 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:40.039+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:40.062 INFO:teuthology.orchestra.run.smithi162.stdout:3753 write 0xe44553d thru 0xe44a773 (0x5237 bytes) 2024-09-16T11:14:40.068 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:40.068 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:40.068 INFO:teuthology.orchestra.run.smithi162.stdout:3756 punch from 0xc132ecf to 0xc13e8cf, (0xba00 bytes) 2024-09-16T11:14:40.069 INFO:teuthology.orchestra.run.smithi162.stdout:3757 write 0x42cda20 thru 0x42d8d41 (0xb322 bytes) 2024-09-16T11:14:40.075 INFO:teuthology.orchestra.run.smithi162.stdout:3758 write 0xca08425 thru 0xca0b8d4 (0x34b0 bytes) 2024-09-16T11:14:40.081 INFO:teuthology.orchestra.run.smithi162.stdout:3759 punch from 0x69f0f4f to 0x69f94eb, (0x859c bytes) 2024-09-16T11:14:40.084 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:40.084 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:40.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:40.094 INFO:teuthology.orchestra.run.smithi162.stdout:3761 trunc from 0xe44a774 to 0x894b82a 2024-09-16T11:14:40.125 INFO:teuthology.orchestra.run.smithi162.stdout:3762 read 0x48d72e7 thru 0x48e1211 (0x9f2b bytes) 2024-09-16T11:14:40.126 INFO:teuthology.orchestra.run.smithi162.stdout:3765 read 0x6b5914b thru 0x6b609a1 (0x7857 bytes) 2024-09-16T11:14:40.127 INFO:teuthology.orchestra.run.smithi162.stdout:3766 punch from 0x6f4b9a to 0x703875, (0xecdb bytes) 2024-09-16T11:14:40.127 INFO:teuthology.orchestra.run.smithi162.stdout:3767 punch from 0x74d120 to 0x751fda, (0x4eba bytes) 2024-09-16T11:14:40.127 INFO:teuthology.orchestra.run.smithi162.stdout:3768 writesame 0x55f3da3 thru 0x55fc0c1 data_size 0x43(0x831f bytes) 2024-09-16T11:14:40.139 INFO:teuthology.orchestra.run.smithi162.stdout:3769 write 0x7e76e1 thru 0x7f3442 (0xbd62 bytes) 2024-09-16T11:14:40.142 INFO:teuthology.orchestra.run.smithi162.stdout:3770 write 0x6488419 thru 0x648a1dc (0x1dc4 bytes) 2024-09-16T11:14:40.142 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:40.137+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:40.148 INFO:teuthology.orchestra.run.smithi162.stdout:3772 read 0x6b7472e thru 0x6b7cb4e (0x8421 bytes) 2024-09-16T11:14:40.148 INFO:teuthology.orchestra.run.smithi162.stdout:3773 write 0x4fb988b thru 0x4fbc16f (0x28e5 bytes) 2024-09-16T11:14:40.173 INFO:teuthology.orchestra.run.smithi162.stdout:3774 trunc from 0x894b82a to 0x882298 2024-09-16T11:14:40.192 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:40.193 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:40.202 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:40.229 INFO:teuthology.orchestra.run.smithi162.stdout:3775 trunc from 0x882298 to 0x80455c1 2024-09-16T11:14:40.238 INFO:teuthology.orchestra.run.smithi162.stdout:3776 write 0x33fe354 thru 0x34029c0 (0x466d bytes) 2024-09-16T11:14:40.243 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:40.239+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:40.245 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:40.257 INFO:teuthology.orchestra.run.smithi162.stdout:3779 write 0x8d83fbd thru 0x8d88e6d (0x4eb1 bytes) 2024-09-16T11:14:40.276 INFO:teuthology.orchestra.run.smithi162.stdout:3783 writesame 0x8f7a65b thru 0x8f881cd data_size 0x1(0xdb73 bytes) 2024-09-16T11:14:40.276 INFO:teuthology.orchestra.run.smithi162.stdout:3784 read 0x5d6345d thru 0x5d71b19 (0xe6bd bytes) 2024-09-16T11:14:40.276 INFO:teuthology.orchestra.run.smithi162.stdout:3785 write 0x292cb2a thru 0x29358a7 (0x8d7e bytes) 2024-09-16T11:14:40.280 INFO:teuthology.orchestra.run.smithi162.stdout:3786 writesame 0x2442660 thru 0x2451c7a data_size 0x1(0xf61b bytes) 2024-09-16T11:14:40.284 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 6, 7, 2] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:40.285 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:40.285 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-16T11:14:40.285 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T11:14:40.285 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T11:14:40.285 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-16T11:14:40.325 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:40.337 INFO:teuthology.orchestra.run.smithi162.stdout:3788 writesame 0xd18d16 thru 0xd225dd data_size 0x4(0x98c8 bytes) 2024-09-16T11:14:40.344 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:40.339+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:40.349 INFO:teuthology.orchestra.run.smithi162.stdout:3790 writesame 0x5954a83 thru 0x59624bd data_size 0x17(0xda3b bytes) 2024-09-16T11:14:40.355 INFO:teuthology.orchestra.run.smithi162.stdout:3791 read 0x2e47569 thru 0x2e4aa26 (0x34be bytes) 2024-09-16T11:14:40.356 INFO:teuthology.orchestra.run.smithi162.stdout:3792 read 0x7d37564 thru 0x7d4419b (0xcc38 bytes) 2024-09-16T11:14:40.356 INFO:teuthology.orchestra.run.smithi162.stdout:3793 read 0x191225d thru 0x191aacf (0x8873 bytes) 2024-09-16T11:14:40.356 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:40.358 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:40.358 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:40.368 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:40.372 INFO:teuthology.orchestra.run.smithi162.stdout:3797 writesame 0xbdf7b6f thru 0xbdfefdf data_size 0x1(0x7471 bytes) 2024-09-16T11:14:40.374 INFO:teuthology.orchestra.run.smithi162.stdout:3798 trunc from 0xbdfefe0 to 0xb1074fc 2024-09-16T11:14:40.388 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:40.388 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "" 2024-09-16T11:14:40.388 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:40.391 INFO:teuthology.orchestra.run.smithi162.stdout:3799 trunc from 0xb1074fc to 0x817395e 2024-09-16T11:14:40.423 INFO:teuthology.orchestra.run.smithi162.stdout:3800 write 0xb3830ce thru 0xb385a17 (0x294a bytes) 2024-09-16T11:14:40.428 INFO:teuthology.orchestra.run.smithi162.stdout:3801 punch from 0x8facf9 to 0x900db0, (0x60b7 bytes) 2024-09-16T11:14:40.428 INFO:teuthology.orchestra.run.smithi162.stdout:3802 write 0xe87b00 thru 0xe8a0e3 (0x25e4 bytes) 2024-09-16T11:14:40.430 INFO:teuthology.orchestra.run.smithi162.stdout:3803 punch from 0xa862a37 to 0xa8688c5, (0x5e8e bytes) 2024-09-16T11:14:40.430 INFO:teuthology.orchestra.run.smithi162.stdout:3806 punch from 0x8c445a to 0x8caa65, (0x660b bytes) 2024-09-16T11:14:40.430 INFO:teuthology.orchestra.run.smithi162.stdout:3807 clone 33 order 24 su 8388608 sc 11 2024-09-16T11:14:40.446 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:40.441+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:40.464 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:40.464 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:40.469 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:40.473 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:40.489 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:40.511 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:14:40.513 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:40.534 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:40.546 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:40.543+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:40.555 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:40.575 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 79.711481762999995, 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:40.578 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:40.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543510114, 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.583 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:40.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:40.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:40.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:40.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:40.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543498552999999, 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.589 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:40.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:40.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543485351000001, 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:40.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543472911000002, 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:40.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:40.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543459400000003, 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.602 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:40.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543448484999999, 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:40.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:40.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543422567, 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.611 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:40.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.615 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543407113000001, 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:40.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:40.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543397227, 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.620 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:40.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.624 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543387463999998, 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:40.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:40.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543377286999998, 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.629 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:40.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:40.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543360804999999, 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:40.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:40.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543346108000001, 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.638 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:40.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543327665, 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:40.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:40.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543310716000001, 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:40.647 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:40.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543291502999999, 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:40.652 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:40.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543274287000003, 2024-09-16T11:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:40.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543253032999999, 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.662 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:40.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.543233065000003, 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:40.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:40.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:40.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:40.671 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:40.672 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:40.673 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:40.674 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:40.674 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:40.675 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:40.675 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:40.667+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:40.773 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:40.769+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:40.784 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:40.784 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:40.784 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:40.784 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:40.791 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:40.875 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:40.869+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:40.890 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:40.891 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:40.891 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:40.891 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:40.891 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:40.900 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:40.975 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:40.969+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:41.000 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.812937279000003, 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.001 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:41.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:41.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.808578752000003, 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:41.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:41.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:41.010 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.804940221999999, 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:41.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.804921436000001, 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:41.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:41.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.804904112999999, 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:41.019 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:41.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:41.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.804886644, 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.024 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:41.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.801046184999997, 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:41.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:41.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.032 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.801036070000002, 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:41.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.036 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.801026188999998, 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:41.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.038 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:41.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.043 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.801017012999999, 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:41.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:41.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.801000946000002, 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:41.048 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:41.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:41.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.800984550999999, 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:41.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:41.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.800974977000003, 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:41.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.800965441000002, 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:41.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:41.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.796908784000003, 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:41.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.796899402000001, 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:41.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.074 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.796890744000002, 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:41.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.796881509999999, 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:41.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:14:41.082 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.103128558000002, 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:14:41.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.4172933039999993, 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.087 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:14:41.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.093 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:41.094 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:41.096 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:41.091+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:41.196 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:41.189+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:41.243 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:41.244 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:41.244 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:41.244 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:41.253 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:41.298 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:41.293+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:41.349 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:41.350 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:41.350 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:41.358 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:41.398 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:41.395+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:41.459 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:41.459 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.679238684, 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:41.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.466 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.291312984, 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.467 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:41.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.472 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.29116165400001, 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:41.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:41.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.291110168, 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.479 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:41.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.29104002, 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:41.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:41.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:41.490 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.29086755, 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:41.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.290809228, 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:41.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:41.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:41.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:41.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:41.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:41.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:41.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:41.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:41.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:41.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:41.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:41.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:41.504 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.505 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.505 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.505 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:41.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.290657108, 2024-09-16T11:14:41.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:41.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:41.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:41.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:41.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:41.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:41.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.290478589, 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.515 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:41.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:41.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.520 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 111.290445616, 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.521 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:41.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:41.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:41.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 110.738257891, 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:41.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 110.738129094, 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.533 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:41.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 110.738070749, 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.539 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:41.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:41.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 110.73801636, 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:41.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:41.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 110.73788205300001, 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.551 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:41.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.956726684, 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:41.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:41.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.562 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.956639372, 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.563 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:41.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.568 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.95640124400001, 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:41.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:41.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.276021235000002, 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:41.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:41.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.567828799000001, 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:14:41.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:14:41.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:41.583 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:41.585 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:41.585 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:41.579+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:41.686 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:41.681+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:41.692 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:41.692 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:41.692 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:41.693 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:41.702 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:41.786 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:41.783+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:41.803 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:41.803 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:41.803 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:41.812 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:41.888 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:41.883+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.738157473999998, 2024-09-16T11:14:41.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:41.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.738079730999999, 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:41.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:41.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:41.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.735713715999999, 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:41.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.930 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.735694199999998, 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:41.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.934 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.734556695999999, 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:41.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:41.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.734540797000001, 2024-09-16T11:14:41.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.734516542999998, 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.944 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.734501283999997, 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.734485964999998, 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:41.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732463561000003, 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.957 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:41.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:41.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:41.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.962 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:41.963 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732449062999997, 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:41.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:41.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732434730000001, 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.969 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:41.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:41.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732420673, 2024-09-16T11:14:41.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:41.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.977 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732406816999998, 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:41.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:41.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.982 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732392941999997, 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:41.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732379174999998, 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732331666, 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.732045847999999, 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.639382032, 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.639232849999999, 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.012 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:42.013 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:42.014 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:42.005+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:42.114 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:42.111+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:42.165 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:42.165 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:42.165 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:42.165 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:42.174 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:42.215 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:42.209+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:42.273 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:42.273 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:42.273 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:42.273 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:42.274 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:42.282 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:42.316 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:42.311+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:42.387 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:42.387 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:42.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:42.387 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.889190315, 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.388 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:42.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:42.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:42.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:42.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:42.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.396 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99422 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 776192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.639405682, 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11283726099999999, 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99422 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742911+0000", 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045312299999999999 2024-09-16T11:14:42.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742922+0000", 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1068e-05 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742939+0000", 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6685e-05 2024-09-16T11:14:42.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742955+0000", 2024-09-16T11:14:42.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6419e-05 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855055+0000", 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11209968100000001 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855150+0000", 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5185000000000004e-05 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855295+0000", 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001451 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99524 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.638310786000002, 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112829479, 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99524 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744021+0000", 2024-09-16T11:14:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046845900000000001 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744026+0000", 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5889999999999996e-06 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744039+0000", 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3326000000000001e-05 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744045+0000", 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9460000000000003e-06 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856169+0000", 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112124052 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856233+0000", 2024-09-16T11:14:42.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4344000000000006e-05 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856382+0000", 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000148763 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99537 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:42.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.635797551, 2024-09-16T11:14:42.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11344878899999999, 2024-09-16T11:14:42.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99537 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:42.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746069+0000", 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746077+0000", 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1440000000000003e-06 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746081+0000", 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3540000000000002e-06 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.857981+0000", 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.111899208 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.858055+0000", 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4374000000000003e-05 2024-09-16T11:14:42.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859515+0000", 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014596030000000001 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:42.413 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.635777517000001, 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:42.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:14:42.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.635508201, 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.420 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:14:42.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.424 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.635109691, 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.425 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:14:42.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:14:42.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:14:42.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.634820903000001, 2024-09-16T11:14:42.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:14:42.432 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.432 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.432 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.433 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:14:42.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.433 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:42.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:14:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:14:42.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:14:42.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:14:42.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:14:42.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:14:42.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:14:42.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:14:42.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:14:42.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:14:42.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:14:42.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:14:42.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.441 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.441 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.441 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99546 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.442 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:42.442 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.633209627999999, 2024-09-16T11:14:42.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11333913599999999, 2024-09-16T11:14:42.442 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.442 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.443 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99546 2024-09-16T11:14:42.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.444 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:42.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:42.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748655+0000", 2024-09-16T11:14:42.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5899999999999995e-07 2024-09-16T11:14:42.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748665+0000", 2024-09-16T11:14:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0776e-05 2024-09-16T11:14:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748714+0000", 2024-09-16T11:14:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8337999999999997e-05 2024-09-16T11:14:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748720+0000", 2024-09-16T11:14:42.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1709999999999999e-06 2024-09-16T11:14:42.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861601+0000", 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112880861 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861743+0000", 2024-09-16T11:14:42.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014191 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861993+0000", 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025022100000000002 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.633121487, 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:42.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:14:42.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:42.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.633043171000001, 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:42.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:14:42.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.462 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.633009432000001, 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.463 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:14:42.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.632968687000002, 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:14:42.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:14:42.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99563 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 630784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.629778245000001, 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112716318, 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99563 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7799999999999997e-07 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752087+0000", 2024-09-16T11:14:42.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.314e-06 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752091+0000", 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752093+0000", 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2170000000000001e-06 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864490+0000", 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11239703700000001 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864596+0000", 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001063 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864801+0000", 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000205038 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.477 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.629768239000001, 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:14:42.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:42.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:42.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:14:42.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:14:42.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.629750181999999, 2024-09-16T11:14:42.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:14:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:14:42.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:14:42.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.492 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.629724193000001, 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:14:42.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.629025762000001, 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.497 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:14:42.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.629013203, 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.502 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:14:42.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99591 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.625323781999999, 2024-09-16T11:14:42.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113533827, 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99591 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756541+0000", 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5200000000000001e-06 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756545+0000", 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756555+0000", 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5070000000000006e-06 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869736+0000", 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11318076000000001 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869888+0000", 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015207900000000001 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870073+0000", 2024-09-16T11:14:42.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018570899999999999 2024-09-16T11:14:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:42.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:42.512 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:42.513 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:42.514 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:42.511+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:42.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:14:42.615 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:42.609+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:42.661 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:42.662 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:42.662 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:42.662 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:42.671 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:42.675 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:42.716 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:42.711+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:42.765 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:42.765 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:42.766 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:42.766 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:42.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:42.802 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:42.816 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:42.811+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:42.823 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:42.851 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:42.869 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:14:42.867+0000 7f70de7fc700 0 --2- 172.21.15.162:0/726443178 >> [v2:172.21.15.162:3301/0,v1:172.21.15.162:6790/0] conn(0x7f70e006bc00 0x7f70e006e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T11:14:42.873 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:42.887 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:42.887 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:42.896 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:42.902 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:42.917 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:42.909+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:42.935 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:42.962 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone32 from 0xb385a18 (overlap 0x882298) to 0x44114c 2024-09-16T11:14:42.970 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:43.001 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:43.003 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:43.004 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:43.012 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.012 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.017 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:43.013+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:43.023 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:43.059 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-16T11:14:43.118 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:43.113+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:43.180 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.180 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.192 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:43.220 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:43.213+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:43.304 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.304 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.314 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:43.320 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:43.315+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:43.421 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:43.413+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:43.424 INFO:teuthology.orchestra.run.smithi162.stdout:3808 write 0xe4a7628 thru 0xe4acf5c (0x5935 bytes) 2024-09-16T11:14:43.427 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.427 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.432 INFO:teuthology.orchestra.run.smithi162.stdout:3809 punch from 0xa45524b to 0xa4629be, (0xd773 bytes) 2024-09-16T11:14:43.433 INFO:teuthology.orchestra.run.smithi162.stdout:3811 read 0xce2a65e thru 0xce2aabe (0x461 bytes) 2024-09-16T11:14:43.433 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:43.433 INFO:teuthology.orchestra.run.smithi162.stdout:3814 write 0x4db89ff thru 0x4dbb0b4 (0x26b6 bytes) 2024-09-16T11:14:43.438 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:43.456 INFO:teuthology.orchestra.run.smithi162.stdout:3815 trunc from 0xe4acf5d to 0x4f16483 2024-09-16T11:14:43.522 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:43.519+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:43.531 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:14:43.546 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.546 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.553 INFO:teuthology.orchestra.run.smithi162.stdout:3821 write 0x640c6de thru 0x641631d (0x9c40 bytes) 2024-09-16T11:14:43.555 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:43.563 INFO:teuthology.orchestra.run.smithi162.stdout:3823 trunc from 0x641631e to 0x11ad0ab 2024-09-16T11:14:43.598 INFO:teuthology.orchestra.run.smithi162.stdout:3824 punch from 0xb22703 to 0xb26c66, (0x4563 bytes) 2024-09-16T11:14:43.620 INFO:teuthology.orchestra.run.smithi162.stdout:3825 write 0x7f5eef4 thru 0x7f6a193 (0xb2a0 bytes) 2024-09-16T11:14:43.622 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:43.619+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:43.644 INFO:teuthology.orchestra.run.smithi162.stdout:3826 write 0xc8c74d2 thru 0xc8d6ff8 (0xfb27 bytes) 2024-09-16T11:14:43.650 INFO:teuthology.orchestra.run.smithi162.stdout:3828 read 0x571d6b3 thru 0x57238cd (0x621b bytes) 2024-09-16T11:14:43.653 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.653 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.665 INFO:teuthology.orchestra.run.smithi162.stdout:3829 write 0xe0230dc thru 0xe02c013 (0x8f38 bytes) 2024-09-16T11:14:43.666 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:43.724 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:43.719+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:43.761 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.761 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.770 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:43.826 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:43.817+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:43.869 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:43.869 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:43.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:43.925 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:43.923+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:44.026 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:44.023+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:44.027 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.027 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.035 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:44.127 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:44.123+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:44.135 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.135 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.145 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:44.228 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:44.221+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:44.240 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.240 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.249 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:44.329 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:44.327+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:44.394 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.394 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.404 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:44.430 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:44.427+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:44.505 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.506 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.514 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:44.531 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:44.527+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:44.554 INFO:teuthology.orchestra.run.smithi162.stdout:3830 punch from 0x132320f to 0x132c80d, (0x95fe bytes) 2024-09-16T11:14:44.554 INFO:teuthology.orchestra.run.smithi162.stdout:3831 read 0xd97710f thru 0xd98244f (0xb341 bytes) 2024-09-16T11:14:44.554 INFO:teuthology.orchestra.run.smithi162.stdout:3832 write 0xba474bc thru 0xba474ea (0x2f bytes) 2024-09-16T11:14:44.559 INFO:teuthology.orchestra.run.smithi162.stdout:3833 write 0x7af083d thru 0x7af2cf3 (0x24b7 bytes) 2024-09-16T11:14:44.562 INFO:teuthology.orchestra.run.smithi162.stdout:3834 write 0x8e423e6 thru 0x8e47771 (0x538c bytes) 2024-09-16T11:14:44.567 INFO:teuthology.orchestra.run.smithi162.stdout:3836 punch from 0xa5b1294 to 0xa5b767e, (0x63ea bytes) 2024-09-16T11:14:44.567 INFO:teuthology.orchestra.run.smithi162.stdout:3837 writesame 0x427c3ea thru 0x42851fb data_size 0x5(0x8e12 bytes) 2024-09-16T11:14:44.599 INFO:teuthology.orchestra.run.smithi162.stdout:3838 trunc from 0xe02c014 to 0x403c325 2024-09-16T11:14:44.616 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.616 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.620 INFO:teuthology.orchestra.run.smithi162.stdout:3839 write 0x71a251 thru 0x729eac (0xfc5c bytes) 2024-09-16T11:14:44.624 INFO:teuthology.orchestra.run.smithi162.stdout:3840 read 0x1c10617 thru 0x1c1d6a0 (0xd08a bytes) 2024-09-16T11:14:44.625 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:44.632 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:44.627+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:44.647 INFO:teuthology.orchestra.run.smithi162.stdout:3841 write 0x8ed8107 thru 0x8ed8cd3 (0xbcd bytes) 2024-09-16T11:14:44.649 INFO:teuthology.orchestra.run.smithi162.stdout:3842 write 0x4d44c70 thru 0x4d51c8b (0xd01c bytes) 2024-09-16T11:14:44.657 INFO:teuthology.orchestra.run.smithi162.stdout:3845 punch from 0x40a57eb to 0x40a8c4a, (0x345f bytes) 2024-09-16T11:14:44.668 INFO:teuthology.orchestra.run.smithi162.stdout:3847 writesame 0x92eb3d2 thru 0x92f0cbd data_size 0x2a(0x58ec bytes) 2024-09-16T11:14:44.688 INFO:teuthology.orchestra.run.smithi162.stdout:3848 trunc from 0x92f0cbe to 0x1678761 2024-09-16T11:14:44.713 INFO:teuthology.orchestra.run.smithi162.stdout:3849 read 0xf4737d thru 0xf5443e (0xd0c2 bytes) 2024-09-16T11:14:44.723 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.723 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.732 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:44.733 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:44.731+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:44.738 INFO:teuthology.orchestra.run.smithi162.stdout:3850 write 0xa5ac3ba thru 0xa5b1eb4 (0x5afb bytes) 2024-09-16T11:14:44.742 INFO:teuthology.orchestra.run.smithi162.stdout:3851 read 0x2901119 thru 0x290744b (0x6333 bytes) 2024-09-16T11:14:44.758 INFO:teuthology.orchestra.run.smithi162.stdout:3854 writesame 0xd60d63a thru 0xd610c19 data_size 0x10(0x35e0 bytes) 2024-09-16T11:14:44.758 INFO:teuthology.orchestra.run.smithi162.stdout:3855 read 0x15247a3 thru 0x1525bbe (0x141c bytes) 2024-09-16T11:14:44.760 INFO:teuthology.orchestra.run.smithi162.stdout:3856 trunc from 0xd610c1a to 0xde8fbbd 2024-09-16T11:14:44.769 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.769 INFO:teuthology.orchestra.run.smithi162.stdout:3858 write 0xd23751c thru 0xd246d07 (0xf7ec bytes) 2024-09-16T11:14:44.775 INFO:teuthology.orchestra.run.smithi162.stdout:3859 read 0x5db3a81 thru 0x5dbe642 (0xabc2 bytes) 2024-09-16T11:14:44.776 INFO:teuthology.orchestra.run.smithi162.stdout:3861 write 0x4672122 thru 0x4678330 (0x620f bytes) 2024-09-16T11:14:44.782 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.783 INFO:teuthology.orchestra.run.smithi162.stdout:3864 punch from 0xda9efef to 0xdaa0220, (0x1231 bytes) 2024-09-16T11:14:44.783 INFO:teuthology.orchestra.run.smithi162.stdout:3867 punch from 0x93af81d to 0x93b550e, (0x5cf1 bytes) 2024-09-16T11:14:44.783 INFO:teuthology.orchestra.run.smithi162.stdout:3868 punch from 0x68138cc to 0x6814537, (0xc6b bytes) 2024-09-16T11:14:44.783 INFO:teuthology.orchestra.run.smithi162.stdout:3869 writesame 0x6b02f4f thru 0x6b0ed44 data_size 0xf(0xbdf6 bytes) 2024-09-16T11:14:44.787 INFO:teuthology.orchestra.run.smithi162.stdout:3870 write 0xcc1b66b thru 0xcc1e67d (0x3013 bytes) 2024-09-16T11:14:44.794 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.794 INFO:teuthology.orchestra.run.smithi162.stdout:3872 write 0x39fd059 thru 0x3a09857 (0xc7ff bytes) 2024-09-16T11:14:44.802 INFO:teuthology.orchestra.run.smithi162.stdout:3873 write 0x4cd6cae thru 0x4ce1b25 (0xae78 bytes) 2024-09-16T11:14:44.805 INFO:teuthology.orchestra.run.smithi162.stdout:3874 read 0xcc68633 thru 0xcc6f934 (0x7302 bytes) 2024-09-16T11:14:44.806 INFO:teuthology.orchestra.run.smithi162.stdout:3876 write 0x6e1ed74 thru 0x6e1ff33 (0x11c0 bytes) 2024-09-16T11:14:44.809 INFO:teuthology.orchestra.run.smithi162.stdout:3877 write 0x4cc6b36 thru 0x4ccb158 (0x4623 bytes) 2024-09-16T11:14:44.813 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.821 INFO:teuthology.orchestra.run.smithi162.stdout:3881 trunc from 0xde8fbbd to 0x9af8875 2024-09-16T11:14:44.834 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:44.831+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:44.837 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.837 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.845 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.845 INFO:teuthology.orchestra.run.smithi162.stdout:3883 write 0x646a4ce thru 0x646fb03 (0x5636 bytes) 2024-09-16T11:14:44.848 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:44.848 INFO:teuthology.orchestra.run.smithi162.stdout:3884 punch from 0x951714f to 0x951ebba, (0x7a6b bytes) 2024-09-16T11:14:44.849 INFO:teuthology.orchestra.run.smithi162.stdout:3886 write 0x4905193 thru 0x4908f8b (0x3df9 bytes) 2024-09-16T11:14:44.851 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.851 INFO:teuthology.orchestra.run.smithi162.stdout:3888 read 0x97866e2 thru 0x97942cc (0xdbeb bytes) 2024-09-16T11:14:44.852 INFO:teuthology.orchestra.run.smithi162.stdout:3890 punch from 0x4a10b84 to 0x4a17998, (0x6e14 bytes) 2024-09-16T11:14:44.863 INFO:teuthology.orchestra.run.smithi162.stdout:3891 trunc from 0x9af8875 to 0x4106ba3 2024-09-16T11:14:44.883 INFO:teuthology.orchestra.run.smithi162.stdout:3892 write 0x3a86293 thru 0x3a9096e (0xa6dc bytes) 2024-09-16T11:14:44.914 INFO:teuthology.orchestra.run.smithi162.stdout:3894 write 0xdf06071 thru 0xdf0f707 (0x9697 bytes) 2024-09-16T11:14:44.921 INFO:teuthology.orchestra.run.smithi162.stdout:3896 read 0xc122f02 thru 0xc12b9c7 (0x8ac6 bytes) 2024-09-16T11:14:44.921 INFO:teuthology.orchestra.run.smithi162.stdout:3897 writesame 0xbec16e1 thru 0xbecbc32 data_size 0x7(0xa552 bytes) 2024-09-16T11:14:44.933 INFO:teuthology.orchestra.run.smithi162.stdout:3898 write 0xacea6da thru 0xacf6c53 (0xc57a bytes) 2024-09-16T11:14:44.934 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:44.929+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:44.941 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.941 INFO:teuthology.orchestra.run.smithi162.stdout:3901 punch from 0x583ff8a to 0x584e4fd, (0xe573 bytes) 2024-09-16T11:14:44.941 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.941 INFO:teuthology.orchestra.run.smithi162.stdout:3903 read 0x1d23437 thru 0x1d2c17d (0x8d47 bytes) 2024-09-16T11:14:44.942 INFO:teuthology.orchestra.run.smithi162.stdout:3904 punch from 0x31501c4 to 0x3156c3d, (0x6a79 bytes) 2024-09-16T11:14:44.942 INFO:teuthology.orchestra.run.smithi162.stdout:3905 read 0x9dc0d4f thru 0x9dce339 (0xd5eb bytes) 2024-09-16T11:14:44.943 INFO:teuthology.orchestra.run.smithi162.stdout:3906 trunc from 0xdf0f708 to 0xecd9b79 2024-09-16T11:14:44.952 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:44.952 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:44.955 INFO:teuthology.orchestra.run.smithi162.stdout:3908 punch from 0x140fb7 to 0x147ae0, (0x6b29 bytes) 2024-09-16T11:14:44.955 INFO:teuthology.orchestra.run.smithi162.stdout:3910 write 0x70712a0 thru 0x7072ad1 (0x1832 bytes) 2024-09-16T11:14:44.963 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:44.963 INFO:teuthology.orchestra.run.smithi162.stdout:3911 write 0x79ea15c thru 0x79f933a (0xf1df bytes) 2024-09-16T11:14:44.969 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:44.969 INFO:teuthology.orchestra.run.smithi162.stdout:3914 read 0x96d4f14 thru 0x96ddc41 (0x8d2e bytes) 2024-09-16T11:14:44.971 INFO:teuthology.orchestra.run.smithi162.stdout:3915 write 0x9ac2af9 thru 0x9acebd3 (0xc0db bytes) 2024-09-16T11:14:44.973 INFO:teuthology.orchestra.run.smithi162.stdout:3916 writesame 0x220b744 thru 0x2213fa1 data_size 0x5(0x885e bytes) 2024-09-16T11:14:44.982 INFO:teuthology.orchestra.run.smithi162.stdout:3917 read 0x83722bb thru 0x8379d87 (0x7acd bytes) 2024-09-16T11:14:44.982 INFO:teuthology.orchestra.run.smithi162.stdout:3918 writesame 0x3d3a3c5 thru 0x3d4097c data_size 0x5d(0x65b8 bytes) 2024-09-16T11:14:44.985 INFO:teuthology.orchestra.run.smithi162.stdout:3919 writesame 0x6338454 thru 0x63460df data_size 0x1e(0xdc8c bytes) 2024-09-16T11:14:44.989 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:45.007 INFO:teuthology.orchestra.run.smithi162.stdout:3925 trunc from 0xecd9b79 to 0x3c2f9da 2024-09-16T11:14:45.017 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:45.037 INFO:teuthology.orchestra.run.smithi162.stdout:3927 read 0x3487a06 thru 0x348aba5 (0x31a0 bytes) 2024-09-16T11:14:45.038 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:45.038 INFO:teuthology.orchestra.run.smithi162.stdout:3929 read 0x2a98fcc thru 0x2aa4853 (0xb888 bytes) 2024-09-16T11:14:45.038 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:45.035+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:45.048 INFO:teuthology.orchestra.run.smithi162.stdout:3931 trunc from 0x3c2f9da to 0x9f9c1e0 2024-09-16T11:14:45.057 INFO:teuthology.orchestra.run.smithi162.stdout:3932 writesame 0x3c8c2db thru 0x3c971b5 data_size 0x2b(0xaedb bytes) 2024-09-16T11:14:45.060 INFO:teuthology.orchestra.run.smithi162.stdout:3934 punch from 0x4ef0854 to 0x4ef2e9d, (0x2649 bytes) 2024-09-16T11:14:45.076 INFO:teuthology.orchestra.run.smithi162.stdout:3935 writesame 0xc17cef1 thru 0xc180c2c data_size 0x2(0x3d3c bytes) 2024-09-16T11:14:45.076 INFO:teuthology.orchestra.run.smithi162.stdout:3936 write 0x5dfe674 thru 0x5e039d5 (0x5362 bytes) 2024-09-16T11:14:45.076 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:45.076 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:45.079 INFO:teuthology.orchestra.run.smithi162.stdout:3937 punch from 0xe53753 to 0xe62f8c, (0xf839 bytes) 2024-09-16T11:14:45.079 INFO:teuthology.orchestra.run.smithi162.stdout:3938 read 0x63e2246 thru 0x63e5968 (0x3723 bytes) 2024-09-16T11:14:45.079 INFO:teuthology.orchestra.run.smithi162.stdout:3939 punch from 0xb0f7b to 0xb781e, (0x68a3 bytes) 2024-09-16T11:14:45.079 INFO:teuthology.orchestra.run.smithi162.stdout:3940 write 0x9f50c8f thru 0x9f57e86 (0x71f8 bytes) 2024-09-16T11:14:45.086 INFO:teuthology.orchestra.run.smithi162.stdout:3941 read 0x425b615 thru 0x425f7b8 (0x41a4 bytes) 2024-09-16T11:14:45.087 INFO:teuthology.orchestra.run.smithi162.stdout:3942 read 0x286ec9e thru 0x287d023 (0xe386 bytes) 2024-09-16T11:14:45.087 INFO:teuthology.orchestra.run.smithi162.stdout:3943 write 0x506a31e thru 0x5074903 (0xa5e6 bytes) 2024-09-16T11:14:45.089 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:45.108 INFO:teuthology.orchestra.run.smithi162.stdout:3944 write 0xe302553 thru 0xe310057 (0xdb05 bytes) 2024-09-16T11:14:45.115 INFO:teuthology.orchestra.run.smithi162.stdout:3947 write 0x56a7616 thru 0x56b57da (0xe1c5 bytes) 2024-09-16T11:14:45.117 INFO:teuthology.orchestra.run.smithi162.stdout:3948 punch from 0xaf0516e to 0xaf0a8dc, (0x576e bytes) 2024-09-16T11:14:45.117 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:45.117 INFO:teuthology.orchestra.run.smithi162.stdout:3951 write 0x785126e thru 0x785b05d (0x9df0 bytes) 2024-09-16T11:14:45.120 INFO:teuthology.orchestra.run.smithi162.stdout:3952 read 0x526d153 thru 0x52785f7 (0xb4a5 bytes) 2024-09-16T11:14:45.121 INFO:teuthology.orchestra.run.smithi162.stdout:3954 read 0x80c98cb thru 0x80d92c5 (0xf9fb bytes) 2024-09-16T11:14:45.137 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:45.129+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:45.137 INFO:teuthology.orchestra.run.smithi162.stdout:3955 trunc from 0xe310058 to 0x5a9de5e 2024-09-16T11:14:45.151 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:45.173 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:45.185 INFO:teuthology.orchestra.run.smithi162.stdout:3956 write 0xd7500e1 thru 0xd757f94 (0x7eb4 bytes) 2024-09-16T11:14:45.192 INFO:teuthology.orchestra.run.smithi162.stdout:3958 write 0x76efcdf thru 0x76f4459 (0x477b bytes) 2024-09-16T11:14:45.192 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:45.195 INFO:teuthology.orchestra.run.smithi162.stdout:3959 punch from 0x2893a77 to 0x289a538, (0x6ac1 bytes) 2024-09-16T11:14:45.195 INFO:teuthology.orchestra.run.smithi162.stdout:3961 write 0x51b7381 thru 0x51b8217 (0xe97 bytes) 2024-09-16T11:14:45.198 INFO:teuthology.orchestra.run.smithi162.stdout:3962 punch from 0x83b5101 to 0x83bfd3c, (0xac3b bytes) 2024-09-16T11:14:45.198 INFO:teuthology.orchestra.run.smithi162.stdout:3964 read 0xa2799dd thru 0xa27fe93 (0x64b7 bytes) 2024-09-16T11:14:45.198 INFO:teuthology.orchestra.run.smithi162.stdout:3966 write 0x72caceb thru 0x72d17e6 (0x6afc bytes) 2024-09-16T11:14:45.200 INFO:teuthology.orchestra.run.smithi162.stdout:3967 read 0xe0018b thru 0xe07662 (0x74d8 bytes) 2024-09-16T11:14:45.201 INFO:teuthology.orchestra.run.smithi162.stdout:3969 punch from 0x84e18f7 to 0x84eb9cb, (0xa0d4 bytes) 2024-09-16T11:14:45.201 INFO:teuthology.orchestra.run.smithi162.stdout:3970 write 0x588475e thru 0x5886353 (0x1bf6 bytes) 2024-09-16T11:14:45.203 INFO:teuthology.orchestra.run.smithi162.stdout:3971 write 0x12566ec thru 0x1264c49 (0xe55e bytes) 2024-09-16T11:14:45.206 INFO:teuthology.orchestra.run.smithi162.stdout:3972 write 0xb16fda2 thru 0xb17dbe5 (0xde44 bytes) 2024-09-16T11:14:45.219 INFO:teuthology.orchestra.run.smithi162.stdout:3973 trunc from 0xd757f95 to 0x9d0e742 2024-09-16T11:14:45.219 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:45.223 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:45.223 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:45.223 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:45.223 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:45.234 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:45.237 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:45.229+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:45.239 INFO:teuthology.orchestra.run.smithi162.stdout:3974 writesame 0x75d4160 thru 0x75e3d4b data_size 0x2e(0xfbec bytes) 2024-09-16T11:14:45.242 INFO:teuthology.orchestra.run.smithi162.stdout:3975 punch from 0x1f4fe2e to 0x1f58b23, (0x8cf5 bytes) 2024-09-16T11:14:45.242 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:45.242 INFO:teuthology.orchestra.run.smithi162.stdout:3979 read 0x311a0b9 thru 0x3121bab (0x7af3 bytes) 2024-09-16T11:14:45.242 INFO:teuthology.orchestra.run.smithi162.stdout:3980 punch from 0x2b2a478 to 0x2b340df, (0x9c67 bytes) 2024-09-16T11:14:45.243 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:45.256 INFO:teuthology.orchestra.run.smithi162.stdout:3981 writesame 0xc3c3175 thru 0xc3d0839 data_size 0x7b(0xd6c5 bytes) 2024-09-16T11:14:45.256 INFO:teuthology.orchestra.run.smithi162.stdout:3982 writesame 0x180f77d thru 0x181770a data_size 0x1d(0x7f8e bytes) 2024-09-16T11:14:45.266 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:45.273 INFO:teuthology.orchestra.run.smithi162.stdout:3984 trunc from 0xc3d083a to 0x3e801e9 2024-09-16T11:14:45.289 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:45.295 INFO:teuthology.orchestra.run.smithi162.stdout:3986 read 0x193ff1d thru 0x1947fd0 (0x80b4 bytes) 2024-09-16T11:14:45.297 INFO:teuthology.orchestra.run.smithi162.stdout:3988 trunc from 0x3e801e9 to 0x4b51433 2024-09-16T11:14:45.311 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:45.314 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:45.315 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:45.330 INFO:teuthology.orchestra.run.smithi162.stdout:3989 write 0xe2669a8 thru 0xe26a4f4 (0x3b4d bytes) 2024-09-16T11:14:45.338 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:45.335+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:45.342 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:45.342 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:45.342 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:45.342 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:45.342 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:45.347 INFO:teuthology.orchestra.run.smithi162.stdout:3990 writesame 0xee55470 thru 0xee646f5 data_size 0x25(0xf286 bytes) 2024-09-16T11:14:45.347 INFO:teuthology.orchestra.run.smithi162.stdout:3991 writesame 0x9a2dbd7 thru 0x9a3118b data_size 0x1(0x35b5 bytes) 2024-09-16T11:14:45.352 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:45.361 INFO:teuthology.orchestra.run.smithi162.stdout:3993 read 0x1774cd6 thru 0x177d5e0 (0x890b bytes) 2024-09-16T11:14:45.362 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:45.362 INFO:teuthology.orchestra.run.smithi162.stdout:3995 write 0x343f05b thru 0x3447ba2 (0x8b48 bytes) 2024-09-16T11:14:45.391 INFO:teuthology.orchestra.run.smithi162.stdout:3996 trunc from 0xee646f6 to 0x10b0a82 2024-09-16T11:14:45.400 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 6, 7, 2] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:45.400 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T11:14:45.440 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:45.433+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:45.443 INFO:teuthology.orchestra.run.smithi162.stdout:3997 writesame 0x59c1f25 thru 0x59c7307 data_size 0x5(0x53e3 bytes) 2024-09-16T11:14:45.445 INFO:teuthology.orchestra.run.smithi162.stdout:3998 write 0x1f1ddb8 thru 0x1f23e19 (0x6062 bytes) 2024-09-16T11:14:45.450 INFO:teuthology.orchestra.run.smithi162.stdout:3999 read 0x5888d6a thru 0x5893129 (0xa3c0 bytes) 2024-09-16T11:14:45.461 INFO:teuthology.orchestra.run.smithi162.stdout:4001 write 0x624b0fd thru 0x62593ff (0xe303 bytes) 2024-09-16T11:14:45.463 INFO:teuthology.orchestra.run.smithi162.stdout:4003 write 0x4678232 thru 0x4680f3f (0x8d0e bytes) 2024-09-16T11:14:45.478 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:45.478 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:45.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:45.478 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.982221756999998, 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:45.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:45.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:45.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99422 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 776192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.732437124, 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11283726099999999, 2024-09-16T11:14:45.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99422 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742911+0000", 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045312299999999999 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742922+0000", 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1068e-05 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742939+0000", 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6685e-05 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742955+0000", 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6419e-05 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855055+0000", 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11209968100000001 2024-09-16T11:14:45.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855150+0000", 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5185000000000004e-05 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855295+0000", 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001451 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.490 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99524 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.731342227999999, 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112829479, 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99524 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744021+0000", 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046845900000000001 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744026+0000", 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5889999999999996e-06 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744039+0000", 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3326000000000001e-05 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744045+0000", 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9460000000000003e-06 2024-09-16T11:14:45.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856169+0000", 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112124052 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856233+0000", 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4344000000000006e-05 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856382+0000", 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000148763 2024-09-16T11:14:45.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99537 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.728828993, 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11344878899999999, 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99537 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746069+0000", 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746077+0000", 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1440000000000003e-06 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746081+0000", 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3540000000000002e-06 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.857981+0000", 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.111899208 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.858055+0000", 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4374000000000003e-05 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859515+0000", 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014596030000000001 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.728808958999998, 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:14:45.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:14:45.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.728539643000001, 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:14:45.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:45.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.728141133000001, 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:14:45.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:14:45.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.727852344999999, 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.513 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:14:45.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.517 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99546 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.72624107, 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11333913599999999, 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99546 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748655+0000", 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5899999999999995e-07 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748665+0000", 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0776e-05 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748714+0000", 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8337999999999997e-05 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748720+0000", 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1709999999999999e-06 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861601+0000", 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112880861 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861743+0000", 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014191 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861993+0000", 2024-09-16T11:14:45.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025022100000000002 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.726152929000001, 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.522 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:14:45.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:14:45.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.726074613000002, 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:14:45.526 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:14:45.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.726040873999999, 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:14:45.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.726000128999999, 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:14:45.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:14:45.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99563 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 630784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.722809687000002, 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112716318, 2024-09-16T11:14:45.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99563 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7799999999999997e-07 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752087+0000", 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.314e-06 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752091+0000", 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752093+0000", 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2170000000000001e-06 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864490+0000", 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11239703700000001 2024-09-16T11:14:45.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864596+0000", 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001063 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864801+0000", 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000205038 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.722799681000001, 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:14:45.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:14:45.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.722781624, 2024-09-16T11:14:45.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:14:45.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.552 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.722755634999999, 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:14:45.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.722057203999999, 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.557 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.561 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.722044645, 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:45.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:14:45.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99591 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.718355224, 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113533827, 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99591 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756541+0000", 2024-09-16T11:14:45.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5200000000000001e-06 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756545+0000", 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756555+0000", 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5070000000000006e-06 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869736+0000", 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11318076000000001 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869888+0000", 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015207900000000001 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870073+0000", 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018570899999999999 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.571 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:45.572 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:14:45.572 INFO:teuthology.orchestra.run.smithi162.stdout:4004 write 0xa0b4bd8 thru 0xa0b52cb (0x6f4 bytes) 2024-09-16T11:14:45.573 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:45.574 INFO:teuthology.orchestra.run.smithi162.stdout:4006 write 0x8e65fb5 thru 0x8e6bf01 (0x5f4d bytes) 2024-09-16T11:14:45.574 INFO:teuthology.orchestra.run.smithi162.stdout:4007 write 0xaf1b1c0 thru 0xaf22754 (0x7595 bytes) 2024-09-16T11:14:45.574 INFO:teuthology.orchestra.run.smithi162.stdout:4009 writesame 0xd1623b7 thru 0xd170f9e data_size 0x4(0xebe8 bytes) 2024-09-16T11:14:45.574 INFO:teuthology.orchestra.run.smithi162.stdout:4010 read 0x12ecf84 thru 0x12f56ee (0x876b bytes) 2024-09-16T11:14:45.574 INFO:teuthology.orchestra.run.smithi162.stdout:4011 read 0x934b539 thru 0x934ce01 (0x18c9 bytes) 2024-09-16T11:14:45.574 INFO:teuthology.orchestra.run.smithi162.stdout:4012 writesame 0xbb0461f thru 0xbb0b7b9 data_size 0x1(0x719b bytes) 2024-09-16T11:14:45.575 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:45.567+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:45.582 INFO:teuthology.orchestra.run.smithi162.stdout:4014 writesame 0xe307b50 thru 0xe316095 data_size 0x1(0xe546 bytes) 2024-09-16T11:14:45.582 INFO:teuthology.orchestra.run.smithi162.stdout:4015 write 0xd02ce3b thru 0xd02f8a5 (0x2a6b bytes) 2024-09-16T11:14:45.614 INFO:teuthology.orchestra.run.smithi162.stdout:4016 trunc from 0xe316096 to 0x84cbf 2024-09-16T11:14:45.660 INFO:teuthology.orchestra.run.smithi162.stdout:4017 punch from 0x6a958 to 0x6d0c0, (0x2768 bytes) 2024-09-16T11:14:45.661 INFO:teuthology.orchestra.run.smithi162.stdout:4018 punch from 0x6af39 to 0x6f324, (0x43eb bytes) 2024-09-16T11:14:45.673 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:45.671+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:45.674 INFO:teuthology.orchestra.run.smithi162.stdout:4019 write 0x2c6aeae thru 0x2c6ced7 (0x202a bytes) 2024-09-16T11:14:45.679 INFO:teuthology.orchestra.run.smithi162.stdout:4020 read 0x25b55d6 thru 0x25ba8df (0x530a bytes) 2024-09-16T11:14:45.680 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:45.680 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:45.680 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:45.680 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:45.691 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:45.692 INFO:teuthology.orchestra.run.smithi162.stdout:4022 writesame 0x52a7629 thru 0x52af66c data_size 0x2(0x8044 bytes) 2024-09-16T11:14:45.692 INFO:teuthology.orchestra.run.smithi162.stdout:4023 punch from 0x24f7f20 to 0x2506196, (0xe276 bytes) 2024-09-16T11:14:45.692 INFO:teuthology.orchestra.run.smithi162.stdout:4024 writesame 0x219ab9 thru 0x22728c data_size 0x26(0xd7d4 bytes) 2024-09-16T11:14:45.697 INFO:teuthology.orchestra.run.smithi162.stdout:4026 writesame 0x20fb358 thru 0x2101e9c data_size 0x1(0x6b45 bytes) 2024-09-16T11:14:45.719 INFO:teuthology.orchestra.run.smithi162.stdout:4028 writesame 0x254a71f thru 0x254cfec data_size 0x3(0x28ce bytes) 2024-09-16T11:14:45.746 INFO:teuthology.orchestra.run.smithi162.stdout:4029 write 0xd7d5dd4 thru 0xd7ddb77 (0x7da4 bytes) 2024-09-16T11:14:45.753 INFO:teuthology.orchestra.run.smithi162.stdout:4030 write 0x2d8612e thru 0x2d8aca2 (0x4b75 bytes) 2024-09-16T11:14:45.755 INFO:teuthology.orchestra.run.smithi162.stdout:4032 punch from 0xba2aebc to 0xba2ffa1, (0x50e5 bytes) 2024-09-16T11:14:45.755 INFO:teuthology.orchestra.run.smithi162.stdout:4033 write 0x90a05f4 thru 0x90a8396 (0x7da3 bytes) 2024-09-16T11:14:45.762 INFO:teuthology.orchestra.run.smithi162.stdout:4035 punch from 0x1605f57 to 0x160ef15, (0x8fbe bytes) 2024-09-16T11:14:45.763 INFO:teuthology.orchestra.run.smithi162.stdout:4036 read 0xa039100 thru 0xa045c4e (0xcb4f bytes) 2024-09-16T11:14:45.763 INFO:teuthology.orchestra.run.smithi162.stdout:4038 read 0xc31698b thru 0xc31b8a0 (0x4f16 bytes) 2024-09-16T11:14:45.763 INFO:teuthology.orchestra.run.smithi162.stdout:4039 read 0x814c40 thru 0x81dc8f (0x9050 bytes) 2024-09-16T11:14:45.763 INFO:teuthology.orchestra.run.smithi162.stdout:4040 read 0x568ea23 thru 0x568f410 (0x9ee bytes) 2024-09-16T11:14:45.763 INFO:teuthology.orchestra.run.smithi162.stdout:4041 write 0x30d068a thru 0x30d4c4f (0x45c6 bytes) 2024-09-16T11:14:45.774 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:45.769+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:45.780 INFO:teuthology.orchestra.run.smithi162.stdout:4042 write 0xe8e11e7 thru 0xe8ed725 (0xc53f bytes) 2024-09-16T11:14:45.787 INFO:teuthology.orchestra.run.smithi162.stdout:4043 writesame 0xd820b3e thru 0xd8287e9 data_size 0x4(0x7cac bytes) 2024-09-16T11:14:45.796 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:45.797 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:45.797 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:45.797 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:45.797 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:45.803 INFO:teuthology.orchestra.run.smithi162.stdout:4044 read 0x75ab26a thru 0x75b8bb6 (0xd94d bytes) 2024-09-16T11:14:45.803 INFO:teuthology.orchestra.run.smithi162.stdout:4045 read 0x69d5574 thru 0x69e18bb (0xc348 bytes) 2024-09-16T11:14:45.804 INFO:teuthology.orchestra.run.smithi162.stdout:4046 punch from 0x74135dd to 0x741a16c, (0x6b8f bytes) 2024-09-16T11:14:45.806 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:45.817 INFO:teuthology.orchestra.run.smithi162.stdout:4048 trunc from 0xe8ed726 to 0x5f5ca50 2024-09-16T11:14:45.860 INFO:teuthology.orchestra.run.smithi162.stdout:4049 writesame 0xd811d7b thru 0xd818ec2 data_size 0x64(0x7148 bytes) 2024-09-16T11:14:45.860 INFO:teuthology.orchestra.run.smithi162.stdout:4050 write 0x668cefe thru 0x668e36a (0x146d bytes) 2024-09-16T11:14:45.867 INFO:teuthology.orchestra.run.smithi162.stdout:4052 trunc from 0xd818ec3 to 0xe9b6b08 2024-09-16T11:14:45.874 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:45.871+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:45.877 INFO:teuthology.orchestra.run.smithi162.stdout:4053 write 0xb9bdf95 thru 0xb9cb143 (0xd1af bytes) 2024-09-16T11:14:45.886 INFO:teuthology.orchestra.run.smithi162.stdout:4054 read 0x8569f65 thru 0x8574a24 (0xaac0 bytes) 2024-09-16T11:14:45.887 INFO:teuthology.orchestra.run.smithi162.stdout:4055 write 0x27a8731 thru 0x27b3f1d (0xb7ed bytes) 2024-09-16T11:14:45.890 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:45.890 INFO:teuthology.orchestra.run.smithi162.stdout:4057 read 0x6ed5f6b thru 0x6ee362c (0xd6c2 bytes) 2024-09-16T11:14:45.910 INFO:teuthology.orchestra.run.smithi162.stdout:4058 trunc from 0xe9b6b08 to 0x2acaa94 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839673455, 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:14:45.921 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:14:45.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:14:45.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:45.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839658997000001, 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:14:45.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:45.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839641796999999, 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:14:45.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839628276999999, 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:14:45.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839619734999999, 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:14:45.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:45.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:14:45.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:14:45.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839609590999999, 2024-09-16T11:14:45.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:14:45.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:45.954 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839596084, 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:45.959 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839582492000002, 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:14:45.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:14:45.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:14:45.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:45.964 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.83955821, 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839524049000001, 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.969 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:14:45.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:14:45.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839490651999999, 2024-09-16T11:14:45.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:14:45.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839468007000001, 2024-09-16T11:14:45.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.984 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839452432000002, 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:14:45.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.839435397999999, 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:14:45.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:14:45.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:14:45.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.835802434000001, 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:45.995 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:45.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:14:45.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:45.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:46.000 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.835768091999999, 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:14:46.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:46.005 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.835752146000001, 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:46.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:14:46.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.835736534999999, 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:14:46.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:14:46.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.835721447000001, 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:46.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:14:46.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.835706644999998, 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:14:46.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:14:46.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.026 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:46.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4059 read 0x147f681 thru 0x1485839 (0x61b9 bytes) 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4060 punch from 0x1b7b490 to 0x1b80d42, (0x58b2 bytes) 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4062 writesame 0xa615f63 thru 0xa61b3df data_size 0x1(0x547d bytes) 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4064 write 0x3342ef1 thru 0x334945f (0x656f bytes) 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4065 read 0x37eeee5 thru 0x37f864d (0x9769 bytes) 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4066 writesame 0x334e5df thru 0x3354630 data_size 0x1(0x6052 bytes) 2024-09-16T11:14:46.029 INFO:teuthology.orchestra.run.smithi162.stdout:4067 trunc from 0xa61b3e0 to 0x8e65c81 2024-09-16T11:14:46.030 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:46.021+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:46.036 INFO:teuthology.orchestra.run.smithi162.stdout:4069 write 0xe9351a4 thru 0xe943038 (0xde95 bytes) 2024-09-16T11:14:46.071 INFO:teuthology.orchestra.run.smithi162.stdout:4070 trunc from 0xe943039 to 0x1042c28 2024-09-16T11:14:46.109 INFO:teuthology.orchestra.run.smithi162.stdout:4071 read 0x65c34b thru 0x661a1f (0x56d5 bytes) 2024-09-16T11:14:46.110 INFO:teuthology.orchestra.run.smithi162.stdout:4072 read 0x100b88 thru 0x1101b5 (0xf62e bytes) 2024-09-16T11:14:46.110 INFO:teuthology.orchestra.run.smithi162.stdout:4073 punch from 0x575fea to 0x57b1bb, (0x51d1 bytes) 2024-09-16T11:14:46.129 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:46.121+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:46.140 INFO:teuthology.orchestra.run.smithi162.stdout:4074 writesame 0xc8c77cc thru 0xc8ccb61 data_size 0xd(0x5396 bytes) 2024-09-16T11:14:46.140 INFO:teuthology.orchestra.run.smithi162.stdout:4075 write 0x49162b2 thru 0x4921d4e (0xba9d bytes) 2024-09-16T11:14:46.148 INFO:teuthology.orchestra.run.smithi162.stdout:4076 write 0x95d384f thru 0x95e0efc (0xd6ae bytes) 2024-09-16T11:14:46.149 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:46.149 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:46.149 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:46.149 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:46.155 INFO:teuthology.orchestra.run.smithi162.stdout:4077 write 0x12563d0 thru 0x125f3cc (0x8ffd bytes) 2024-09-16T11:14:46.158 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:46.177 INFO:teuthology.orchestra.run.smithi162.stdout:4079 write 0xe5c95c4 thru 0xe5d3ee7 (0xa924 bytes) 2024-09-16T11:14:46.184 INFO:teuthology.orchestra.run.smithi162.stdout:4080 read 0x8cd07bb thru 0x8cd526a (0x4ab0 bytes) 2024-09-16T11:14:46.188 INFO:teuthology.orchestra.run.smithi162.stdout:4081 trunc from 0xe5d3ee8 to 0xc709f82 2024-09-16T11:14:46.214 INFO:teuthology.orchestra.run.smithi162.stdout:4082 trunc from 0xc709f82 to 0xa1bfe4c 2024-09-16T11:14:46.230 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:46.227+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:46.235 INFO:teuthology.orchestra.run.smithi162.stdout:4083 trunc from 0xa1bfe4c to 0x69d1855 2024-09-16T11:14:46.247 INFO:teuthology.orchestra.run.smithi162.stdout:4085 write 0x35e0dbc thru 0x35e47e7 (0x3a2c bytes) 2024-09-16T11:14:46.253 INFO:teuthology.orchestra.run.smithi162.stdout:4086 write 0x380081e thru 0x38017c5 (0xfa8 bytes) 2024-09-16T11:14:46.256 INFO:teuthology.orchestra.run.smithi162.stdout:4087 writesame 0x67d265f thru 0x67d3227 data_size 0x1(0xbc9 bytes) 2024-09-16T11:14:46.262 INFO:teuthology.orchestra.run.smithi162.stdout:4088 trunc from 0x69d1855 to 0x81e052f 2024-09-16T11:14:46.274 INFO:teuthology.orchestra.run.smithi162.stdout:4090 writesame 0x499f9ff thru 0x49a03e2 data_size 0x6(0x9e4 bytes) 2024-09-16T11:14:46.288 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:46.289 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:46.289 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:46.289 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:46.289 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:46.299 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:46.302 INFO:teuthology.orchestra.run.smithi162.stdout:4092 writesame 0xebc4897 thru 0xebd00a7 data_size 0x3(0xb811 bytes) 2024-09-16T11:14:46.302 INFO:teuthology.orchestra.run.smithi162.stdout:4093 write 0x1013619 thru 0x1014371 (0xd59 bytes) 2024-09-16T11:14:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:4095 punch from 0xa421a9 to 0xa48ba6, (0x69fd bytes) 2024-09-16T11:14:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:4096 read 0xc73694e thru 0xc736f3c (0x5ef bytes) 2024-09-16T11:14:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:4097 punch from 0x3680c5f to 0x368c9bb, (0xbd5c bytes) 2024-09-16T11:14:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:46.305 INFO:teuthology.orchestra.run.smithi162.stdout:4100 writesame 0x53d6376 thru 0x53dffa3 data_size 0x1(0x9c2e bytes) 2024-09-16T11:14:46.331 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:46.327+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:46.342 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:46.342 INFO:teuthology.orchestra.run.smithi162.stdout:4102 punch from 0xc30a9e1 to 0xc30d496, (0x2ab5 bytes) 2024-09-16T11:14:46.342 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:46.342 INFO:teuthology.orchestra.run.smithi162.stdout:4104 read 0x42113f1 thru 0x4213ca3 (0x28b3 bytes) 2024-09-16T11:14:46.342 INFO:teuthology.orchestra.run.smithi162.stdout:4105 read 0x6620744 thru 0x6627e7d (0x773a bytes) 2024-09-16T11:14:46.343 INFO:teuthology.orchestra.run.smithi162.stdout:4107 read 0x9a1a06d thru 0x9a21fed (0x7f81 bytes) 2024-09-16T11:14:46.343 INFO:teuthology.orchestra.run.smithi162.stdout:4109 write 0x8697011 thru 0x86a4785 (0xd775 bytes) 2024-09-16T11:14:46.351 INFO:teuthology.orchestra.run.smithi162.stdout:4110 write 0x1332926 thru 0x133ca5c (0xa137 bytes) 2024-09-16T11:14:46.354 INFO:teuthology.orchestra.run.smithi162.stdout:4112 writesame 0xd31a487 thru 0xd323ddc data_size 0x13(0x9956 bytes) 2024-09-16T11:14:46.362 INFO:teuthology.orchestra.run.smithi162.stdout:4113 write 0xdf277f0 thru 0xdf343a5 (0xcbb6 bytes) 2024-09-16T11:14:46.368 INFO:teuthology.orchestra.run.smithi162.stdout:4114 punch from 0x1ab4734 to 0x1ac28d6, (0xe1a2 bytes) 2024-09-16T11:14:46.368 INFO:teuthology.orchestra.run.smithi162.stdout:4115 read 0x31b9dad thru 0x31c233e (0x8592 bytes) 2024-09-16T11:14:46.369 INFO:teuthology.orchestra.run.smithi162.stdout:4116 read 0xb991242 thru 0xb997279 (0x6038 bytes) 2024-09-16T11:14:46.370 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:46.370 INFO:teuthology.orchestra.run.smithi162.stdout:4118 write 0xabe0962 thru 0xabe9d22 (0x93c1 bytes) 2024-09-16T11:14:46.373 INFO:teuthology.orchestra.run.smithi162.stdout:4120 punch from 0x40b45fa to 0x40bf767, (0xb16d bytes) 2024-09-16T11:14:46.373 INFO:teuthology.orchestra.run.smithi162.stdout:4121 write 0x8e28893 thru 0x8e33479 (0xabe7 bytes) 2024-09-16T11:14:46.376 INFO:teuthology.orchestra.run.smithi162.stdout:4123 write 0x9bab2e2 thru 0x9bb3f3b (0x8c5a bytes) 2024-09-16T11:14:46.383 INFO:teuthology.orchestra.run.smithi162.stdout:4124 writesame 0x831f86a thru 0x8324e47 data_size 0x3a(0x55de bytes) 2024-09-16T11:14:46.385 INFO:teuthology.orchestra.run.smithi162.stdout:4125 write 0x1e60bc8 thru 0x1e61316 (0x74f bytes) 2024-09-16T11:14:46.393 INFO:teuthology.orchestra.run.smithi162.stdout:4126 punch from 0x53c79c4 to 0x53c82ce, (0x90a bytes) 2024-09-16T11:14:46.393 INFO:teuthology.orchestra.run.smithi162.stdout:4128 read 0xa11387a thru 0xa11a7cd (0x6f54 bytes) 2024-09-16T11:14:46.409 INFO:teuthology.orchestra.run.smithi162.stdout:4129 trunc from 0xebd00a8 to 0x4cf56de 2024-09-16T11:14:46.412 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.275577689000002, 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:46.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.275559301000001, 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:46.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:46.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.422 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.275258544000003, 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:46.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.275228839, 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.427 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:46.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:46.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:46.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:46.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:46.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.275215441, 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.433 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:46.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.275201637000002, 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.443 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.274826638, 2024-09-16T11:14:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.444 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:46.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:46.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:46.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:46.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:46.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.261282606999998, 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:46.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.261261032999997, 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:46.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.261244533999999, 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.466 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.261228457999998, 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:46.472 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.260844884000001, 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:46.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.248762706999997, 2024-09-16T11:14:46.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:46.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:46.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:46.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:46.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:46.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:46.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:46.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.248754505999997, 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:46.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:46.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:46.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:46.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.248746484000002, 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:46.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:46.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:46.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 29.565267604999999, 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:14:46.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:14:46.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:14:46.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:46.503 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.331849254000002, 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.504 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:14:46.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.510 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.945455881000001, 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.511 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:14:46.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 12.993851233000001, 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:14:46.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:14:46.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 5.8580543010000001, 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:14:46.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:46.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:14:46.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:46.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:46.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.524 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:14:46.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:46.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:14:46.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:14:46.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:46.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:46.534 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:46.536 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:46.537 INFO:teuthology.orchestra.run.smithi162.stdout:4130 trunc from 0x4cf56de to 0x67be7d0 2024-09-16T11:14:46.537 INFO:teuthology.orchestra.run.smithi162.stdout:4132 write 0xc6c58f1 thru 0xc6d0e33 (0xb543 bytes) 2024-09-16T11:14:46.537 INFO:teuthology.orchestra.run.smithi162.stdout:4133 read 0xa9b6ef0 thru 0xa9ba962 (0x3a73 bytes) 2024-09-16T11:14:46.537 INFO:teuthology.orchestra.run.smithi162.stdout:4134 write 0xce88df1 thru 0xce92928 (0x9b38 bytes) 2024-09-16T11:14:46.537 INFO:teuthology.orchestra.run.smithi162.stdout:4135 write 0xc53ecdd thru 0xc5416d9 (0x29fd bytes) 2024-09-16T11:14:46.537 INFO:teuthology.orchestra.run.smithi162.stdout:4136 trunc from 0xce92929 to 0xddd05bb 2024-09-16T11:14:46.538 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:46.529+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:46.557 INFO:teuthology.orchestra.run.smithi162.stdout:4137 trunc from 0xddd05bb to 0xf143c1 2024-09-16T11:14:46.579 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T11:14:46.585 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:14:46.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T11:14:46.608 INFO:teuthology.orchestra.run.smithi162.stdout:4139 read 0x620c98 thru 0x6308a9 (0xfc12 bytes) 2024-09-16T11:14:46.609 INFO:teuthology.orchestra.run.smithi162.stdout:4140 punch from 0xac7057 to 0xacd823, (0x67cc bytes) 2024-09-16T11:14:46.631 INFO:teuthology.orchestra.run.smithi162.stdout:4141 trunc from 0xf143c1 to 0xd5cc98d 2024-09-16T11:14:46.636 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:46.631+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:46.639 INFO:teuthology.orchestra.run.smithi162.stdout:4142 read 0x2fcd99 thru 0x30620c (0x9474 bytes) 2024-09-16T11:14:46.640 INFO:teuthology.orchestra.run.smithi162.stdout:4143 punch from 0x709e734 to 0x70ae5b6, (0xfe82 bytes) 2024-09-16T11:14:46.651 INFO:teuthology.orchestra.run.smithi162.stdout:4144 trunc from 0xd5cc98d to 0x6e27554 2024-09-16T11:14:46.653 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:46.653 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:46.653 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:46.660 INFO:teuthology.orchestra.run.smithi162.stdout:4146 punch from 0x30917d7 to 0x3094cd3, (0x34fc bytes) 2024-09-16T11:14:46.663 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:46.674 INFO:teuthology.orchestra.run.smithi162.stdout:4149 writesame 0x8d91e15 thru 0x8d957ad data_size 0x5(0x3999 bytes) 2024-09-16T11:14:46.674 INFO:teuthology.orchestra.run.smithi162.stdout:4150 punch from 0x44f3bda to 0x4502c1e, (0xf044 bytes) 2024-09-16T11:14:46.675 INFO:teuthology.orchestra.run.smithi162.stdout:4152 punch from 0xb1a7d4 to 0xb258a3, (0xb0cf bytes) 2024-09-16T11:14:46.675 INFO:teuthology.orchestra.run.smithi162.stdout:4153 read 0x1295343 thru 0x12a1129 (0xbde7 bytes) 2024-09-16T11:14:46.675 INFO:teuthology.orchestra.run.smithi162.stdout:4157 writesame 0x21ab1bb thru 0x21b44f0 data_size 0x21(0x9336 bytes) 2024-09-16T11:14:46.702 INFO:teuthology.orchestra.run.smithi162.stdout:4158 write 0xeb99fd2 thru 0xeba4ae5 (0xab14 bytes) 2024-09-16T11:14:46.709 INFO:teuthology.orchestra.run.smithi162.stdout:4159 write 0x4f1e8a2 thru 0x4f222f4 (0x3a53 bytes) 2024-09-16T11:14:46.716 INFO:teuthology.orchestra.run.smithi162.stdout:4160 write 0x97ceb00 thru 0x97d2bf3 (0x40f4 bytes) 2024-09-16T11:14:46.721 INFO:teuthology.orchestra.run.smithi162.stdout:4162 read 0x4b5397e thru 0x4b5e424 (0xaaa7 bytes) 2024-09-16T11:14:46.741 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:46.735+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:46.745 INFO:teuthology.orchestra.run.smithi162.stdout:4163 trunc from 0xeba4ae6 to 0x1aca9fb 2024-09-16T11:14:46.772 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:46.772 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:46.772 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:46.772 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:46.772 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:46.773 INFO:teuthology.orchestra.run.smithi162.stdout:4164 writesame 0x165099e thru 0x165736d data_size 0x8(0x69d0 bytes) 2024-09-16T11:14:46.781 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:46.782 INFO:teuthology.orchestra.run.smithi162.stdout:4165 punch from 0x11fdfe8 to 0x1209f71, (0xbf89 bytes) 2024-09-16T11:14:46.782 INFO:teuthology.orchestra.run.smithi162.stdout:4166 trunc from 0x1aca9fb to 0x1bcf0db 2024-09-16T11:14:46.814 INFO:teuthology.orchestra.run.smithi162.stdout:4169 write 0x995612c thru 0x99638b1 (0xd786 bytes) 2024-09-16T11:14:46.830 INFO:teuthology.orchestra.run.smithi162.stdout:4170 write 0xa2ff05d thru 0xa303055 (0x3ff9 bytes) 2024-09-16T11:14:46.836 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:46.837 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:46.833+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:46.841 INFO:teuthology.orchestra.run.smithi162.stdout:4173 trunc from 0xa303056 to 0xcabe3f3 2024-09-16T11:14:46.855 INFO:teuthology.orchestra.run.smithi162.stdout:4174 trunc from 0xcabe3f3 to 0xaa7359d 2024-09-16T11:14:46.867 INFO:teuthology.orchestra.run.smithi162.stdout:4175 read 0x672daea thru 0x6736e88 (0x939f bytes) 2024-09-16T11:14:46.879 INFO:teuthology.orchestra.run.smithi162.stdout:4176 write 0xbb69b36 thru 0xbb72c60 (0x912b bytes) 2024-09-16T11:14:46.888 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:46.888 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.81015416, 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.889 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:46.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.810134794, 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:46.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.810094031999999, 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.900 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:46.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.904 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.810063234000001, 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:46.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:14:46.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.810044646000001, 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:14:46.910 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:46.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.810026594, 2024-09-16T11:14:46.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:46.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.810006667, 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:46.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:46.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.809987828000001, 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:46.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.809795878999999, 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.931 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:14:46.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.936 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.937 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.937 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:46.938 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.809776598999999, 2024-09-16T11:14:46.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:14:46.938 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:46.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:14:46.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806306333999999, 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.945 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:46.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:14:46.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:46.950 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806285031000002, 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.951 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:46.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.952 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:46.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:14:46.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.80627496, 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.958 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:46.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.959 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:46.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:14:46.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806246656999999, 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.964 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:46.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:46.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.969 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806232728000001, 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:46.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:46.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806223965000001, 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.975 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:46.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:14:46.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806200145999998, 2024-09-16T11:14:46.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:14:46.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806169023999999, 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:14:46.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:14:46.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806121285, 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.991 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:46.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:14:46.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 24.806099177, 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:46.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:14:46.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:14:46.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.001 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:47.003 INFO:teuthology.orchestra.run.smithi162.stdout:4177 trunc from 0xbb72c61 to 0x7a0b999 2024-09-16T11:14:47.003 INFO:teuthology.orchestra.run.smithi162.stdout:4178 writesame 0x9dd093 thru 0x9ecd51 data_size 0x1(0xfcbf bytes) 2024-09-16T11:14:47.003 INFO:teuthology.orchestra.run.smithi162.stdout:4180 punch from 0x5767bb9 to 0x5773535, (0xb97c bytes) 2024-09-16T11:14:47.003 INFO:teuthology.orchestra.run.smithi162.stdout:4182 trunc from 0x7a0b999 to 0x577438 2024-09-16T11:14:47.004 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:46.999+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:47.004 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:47.004 INFO:teuthology.orchestra.run.smithi162.stdout:4183 punch from 0x3c2d35 to 0x3cec99, (0xbf64 bytes) 2024-09-16T11:14:47.022 INFO:teuthology.orchestra.run.smithi162.stdout:4184 trunc from 0x577438 to 0xa9a0fdf 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4185 read 0xa54b031 thru 0xa553b16 (0x8ae6 bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4187 read 0x6f0d8cc thru 0x6f1540a (0x7b3f bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4188 read 0x44d402f thru 0x44d9493 (0x5465 bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4189 read 0x12ef3ff thru 0x12f1cb9 (0x28bb bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4191 read 0x4b226eb thru 0x4b2e412 (0xbd28 bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4192 read 0x23910f2 thru 0x2393098 (0x1fa7 bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4193 read 0x88ba003 thru 0x88c1bee (0x7bec bytes) 2024-09-16T11:14:47.032 INFO:teuthology.orchestra.run.smithi162.stdout:4195 punch from 0x1680f7f to 0x1681d21, (0xda2 bytes) 2024-09-16T11:14:47.033 INFO:teuthology.orchestra.run.smithi162.stdout:4196 read 0x7d82658 thru 0x7d8f3a0 (0xcd49 bytes) 2024-09-16T11:14:47.033 INFO:teuthology.orchestra.run.smithi162.stdout:4197 read 0x7d92c77 thru 0x7d9a624 (0x79ae bytes) 2024-09-16T11:14:47.035 INFO:teuthology.orchestra.run.smithi162.stdout:4199 trunc from 0xa9a0fdf to 0x89d2468 2024-09-16T11:14:47.058 INFO:teuthology.orchestra.run.smithi162.stdout:4200 trunc from 0x89d2468 to 0xd875cb9 2024-09-16T11:14:47.082 INFO:teuthology.orchestra.run.smithi162.stdout:4203 trunc from 0xd875cb9 to 0x7777338 2024-09-16T11:14:47.094 INFO:teuthology.orchestra.run.smithi162.stdout:4205 read 0x2716ed0 thru 0x271f3ac (0x84dd bytes) 2024-09-16T11:14:47.104 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:47.099+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:47.111 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:47.111 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:47.112 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:47.112 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:47.119 INFO:teuthology.orchestra.run.smithi162.stdout:4206 write 0xc9326da thru 0xc939894 (0x71bb bytes) 2024-09-16T11:14:47.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:47.125 INFO:teuthology.orchestra.run.smithi162.stdout:4208 read 0x35c7924 thru 0x35d5069 (0xd746 bytes) 2024-09-16T11:14:47.147 INFO:teuthology.orchestra.run.smithi162.stdout:4209 trunc from 0xc939895 to 0x229b281 2024-09-16T11:14:47.171 INFO:teuthology.orchestra.run.smithi162.stdout:4210 read 0x107aad3 thru 0x107ed0d (0x423b bytes) 2024-09-16T11:14:47.187 INFO:teuthology.orchestra.run.smithi162.stdout:4211 trunc from 0x229b281 to 0xa3047f0 2024-09-16T11:14:47.201 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:47.201 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:47.202 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:47.202 INFO:teuthology.orchestra.run.smithi162.stdout:4215 punch from 0x5504b5e to 0x5510750, (0xbbf2 bytes) 2024-09-16T11:14:47.205 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:47.203+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:47.206 INFO:teuthology.orchestra.run.smithi162.stdout:4217 trunc from 0xa3047f0 to 0xc67cf29 2024-09-16T11:14:47.217 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:47.218 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:47.218 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:47.218 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:47.218 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:47.218 INFO:teuthology.orchestra.run.smithi162.stdout:4218 punch from 0x7b6e41c to 0x7b7d552, (0xf136 bytes) 2024-09-16T11:14:47.218 INFO:teuthology.orchestra.run.smithi162.stdout:4219 write 0xb158b0a thru 0xb15b76d (0x2c64 bytes) 2024-09-16T11:14:47.225 INFO:teuthology.orchestra.run.smithi162.stdout:4221 read 0x5d28ac0 thru 0x5d383af (0xf8f0 bytes) 2024-09-16T11:14:47.227 INFO:teuthology.orchestra.run.smithi162.stdout:4222 trunc from 0xc67cf29 to 0xc4df2e6 2024-09-16T11:14:47.228 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:47.251 INFO:teuthology.orchestra.run.smithi162.stdout:4224 write 0xce1776e thru 0xce191c6 (0x1a59 bytes) 2024-09-16T11:14:47.258 INFO:teuthology.orchestra.run.smithi162.stdout:4225 writesame 0x5044168 thru 0x504fd0b data_size 0x6(0xbba4 bytes) 2024-09-16T11:14:47.282 INFO:teuthology.orchestra.run.smithi162.stdout:4226 trunc from 0xce191c7 to 0x87bec56 2024-09-16T11:14:47.306 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:47.303+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:47.306 INFO:teuthology.orchestra.run.smithi162.stdout:4228 punch from 0x701febb to 0x7028bac, (0x8cf1 bytes) 2024-09-16T11:14:47.307 INFO:teuthology.orchestra.run.smithi162.stdout:4229 writesame 0xb1dc74 thru 0xb2b284 data_size 0x3(0xd611 bytes) 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:47.325 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.46038987, 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:47.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:47.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292418220999998, 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.330 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:47.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.336 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292406659999997, 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.341 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292393457999999, 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:47.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:47.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:47.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:47.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292381018, 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:47.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:47.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:47.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:47.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:47.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:47.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:47.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292367507000002, 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292356591999997, 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.358 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.362 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292330673999999, 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:47.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292315219999999, 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:47.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:47.370 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292305333999998, 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:47.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:47.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292295570999997, 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.375 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:47.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292285393999997, 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:47.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:47.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:47.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292268911999997, 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.384 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.388 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292254215, 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:47.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:47.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292235771999998, 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.393 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292218822999999, 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.397 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:47.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292199609999997, 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.402 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292182394000001, 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.406 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:47.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292161139999997, 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.292141172000001, 2024-09-16T11:14:47.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:47.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:47.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:47.420 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:47.420 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:47.420 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:47.422 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:47.423 INFO:teuthology.orchestra.run.smithi162.stdout:4230 writesame 0x78f572 thru 0x7918d0 data_size 0x1(0x235f bytes) 2024-09-16T11:14:47.423 INFO:teuthology.orchestra.run.smithi162.stdout:4231 read 0x41d5057 thru 0x41e4bcd (0xfb77 bytes) 2024-09-16T11:14:47.423 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4233 read 0x453028 thru 0x46212f (0xf108 bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4234 write 0x73d5c69 thru 0x73dfea2 (0xa23a bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4237 write 0x26474fa thru 0x26555bf (0xe0c6 bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4238 writesame 0xe3fad55 thru 0xe3fd5f8 data_size 0x9(0x28a4 bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4240 read 0xc567b6 thru 0xc646ac (0xdef7 bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4242 writesame 0xab2f2c9 thru 0xab3a346 data_size 0x3a(0xb07e bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4244 writesame 0x9afdea0 thru 0x9b06cff data_size 0x44(0x8e60 bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4245 write 0x8acb7bc thru 0x8ad11c1 (0x5a06 bytes) 2024-09-16T11:14:47.424 INFO:teuthology.orchestra.run.smithi162.stdout:4246 trunc from 0xe3fd5f9 to 0x7372339 2024-09-16T11:14:47.425 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:47.419+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:47.445 INFO:teuthology.orchestra.run.smithi162.stdout:4247 trunc from 0x7372339 to 0xdaaee04 2024-09-16T11:14:47.454 INFO:teuthology.orchestra.run.smithi162.stdout:4248 writesame 0x400f208 thru 0x40167d8 data_size 0x1(0x75d1 bytes) 2024-09-16T11:14:47.478 INFO:teuthology.orchestra.run.smithi162.stdout:4249 read 0xc6dd397 thru 0xc6e8e3c (0xbaa6 bytes) 2024-09-16T11:14:47.495 INFO:teuthology.orchestra.run.smithi162.stdout:4250 trunc from 0xdaaee04 to 0x2b962d7 2024-09-16T11:14:47.522 INFO:teuthology.orchestra.run.smithi162.stdout:4251 write 0x1ff7b39 thru 0x1ffb498 (0x3960 bytes) 2024-09-16T11:14:47.522 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:47.519+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:47.525 INFO:teuthology.orchestra.run.smithi162.stdout:4252 read 0x5552a9 thru 0x5579a7 (0x26ff bytes) 2024-09-16T11:14:47.534 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:47.534 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:47.534 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:47.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:47.548 INFO:teuthology.orchestra.run.smithi162.stdout:4253 write 0xa2c56f5 thru 0xa2cf94d (0xa259 bytes) 2024-09-16T11:14:47.557 INFO:teuthology.orchestra.run.smithi162.stdout:4254 write 0x8870153 thru 0x887e6db (0xe589 bytes) 2024-09-16T11:14:47.559 INFO:teuthology.orchestra.run.smithi162.stdout:4255 punch from 0x6d73fc4 to 0x6d77ac0, (0x3afc bytes) 2024-09-16T11:14:47.560 INFO:teuthology.orchestra.run.smithi162.stdout:4256 read 0x363723 thru 0x36c75c (0x903a bytes) 2024-09-16T11:14:47.565 INFO:teuthology.orchestra.run.smithi162.stdout:4257 trunc from 0xa2cf94e to 0x76c81ce 2024-09-16T11:14:47.565 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:47.586 INFO:teuthology.orchestra.run.smithi162.stdout:4262 punch from 0x57d280b to 0x57d86c3, (0x5eb8 bytes) 2024-09-16T11:14:47.587 INFO:teuthology.orchestra.run.smithi162.stdout:4263 read 0x758acb3 thru 0x758c86b (0x1bb9 bytes) 2024-09-16T11:14:47.588 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T11:14:47.590 INFO:teuthology.orchestra.run.smithi162.stdout:4264 read 0x179c0bb thru 0x17a6d9e (0xace4 bytes) 2024-09-16T11:14:47.592 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:47.603 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T11:14:47.608 INFO:teuthology.orchestra.run.smithi162.stdout:4266 write 0xc664fc0 thru 0xc672060 (0xd0a1 bytes) 2024-09-16T11:14:47.614 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:47.615 INFO:teuthology.orchestra.run.smithi162.stdout:4269 writesame 0x68858b2 thru 0x6890031 data_size 0x86(0xa780 bytes) 2024-09-16T11:14:47.621 INFO:teuthology.orchestra.run.smithi162.stdout:4270 writesame 0x222cc3e thru 0x222e8fd data_size 0x28(0x1cc0 bytes) 2024-09-16T11:14:47.623 INFO:teuthology.orchestra.run.smithi162.stdout:4271 write 0x9273ab5 thru 0x9275a93 (0x1fdf bytes) 2024-09-16T11:14:47.624 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:47.617+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:47.628 INFO:teuthology.orchestra.run.smithi162.stdout:4272 write 0xad2f2 thru 0xaee1e (0x1b2d bytes) 2024-09-16T11:14:47.631 INFO:teuthology.orchestra.run.smithi162.stdout:4273 writesame 0x8636428 thru 0x86388bd data_size 0x15(0x2496 bytes) 2024-09-16T11:14:47.633 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:47.636 INFO:teuthology.orchestra.run.smithi162.stdout:4274 read 0x6c09bd thru 0x6cb990 (0xafd4 bytes) 2024-09-16T11:14:47.637 INFO:teuthology.orchestra.run.smithi162.stdout:4275 write 0x55550b0 thru 0x555e6d5 (0x9626 bytes) 2024-09-16T11:14:47.652 INFO:teuthology.orchestra.run.smithi162.stdout:4276 trunc from 0xc672061 to 0x7ef3db1 2024-09-16T11:14:47.654 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:47.658 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:47.658 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:47.658 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:47.658 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:47.658 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:47.668 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:47.674 INFO:teuthology.orchestra.run.smithi162.stdout:4277 read 0x2a98bdf thru 0x2aa2580 (0x99a2 bytes) 2024-09-16T11:14:47.674 INFO:teuthology.orchestra.run.smithi162.stdout:4279 clone 34 order 23 su 32768 sc 13 2024-09-16T11:14:47.678 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:47.697 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:47.722 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:47.724 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:47.719+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:47.725 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:47.726 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:47.758 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:47.847 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:47.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:47.850 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:47.845+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:47.950 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:47.947+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:47.952 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:47.952 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:47.961 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:48.051 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:48.045+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:48.057 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.057 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.067 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:48.152 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:48.145+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:48.164 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.164 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:48.253 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:48.249+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:48.281 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.281 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.291 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:48.354 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:48.349+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:48.389 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.389 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.395 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:48.455 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:48.451+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:48.492 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.493 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.501 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:48.556 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:48.549+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:48.598 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:14:48.601 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.601 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.602 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T11:14:48.611 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:48.616 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T11:14:48.658 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:48.655+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:48.717 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone33 from 0x7ef3db1 (overlap 0x84cbf) to 0x0 2024-09-16T11:14:48.758 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:48.755+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:48.786 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.786 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.802 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:48.833 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-16T11:14:48.859 INFO:teuthology.orchestra.run.smithi162.stdout:4281 read 0x3ff994d thru 0x4009781 (0xfe35 bytes) 2024-09-16T11:14:48.859 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:48.855+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:48.883 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:48.883 INFO:teuthology.orchestra.run.smithi162.stdout:4284 read 0x7ad05a0 thru 0x7ad6e94 (0x68f5 bytes) 2024-09-16T11:14:48.929 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:48.929 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:48.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:48.977 INFO:teuthology.orchestra.run.smithi162.stdout:4285 write 0x8abd198 thru 0x8accef1 (0xfd5a bytes) 2024-09-16T11:14:48.978 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:48.975+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:49.006 INFO:teuthology.orchestra.run.smithi162.stdout:4286 write 0x218547f thru 0x2188c73 (0x37f5 bytes) 2024-09-16T11:14:49.007 INFO:teuthology.orchestra.run.smithi162.stdout:4287 read 0x4b54b71 thru 0x4b55702 (0xb92 bytes) 2024-09-16T11:14:49.010 INFO:teuthology.orchestra.run.smithi162.stdout:4288 read 0x37bd753 thru 0x37c5f4f (0x87fd bytes) 2024-09-16T11:14:49.061 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:49.059+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:49.100 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.100 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.109 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:49.110 INFO:teuthology.orchestra.run.smithi162.stdout:4289 write 0xa73f0c5 thru 0xa74bf15 (0xce51 bytes) 2024-09-16T11:14:49.129 INFO:teuthology.orchestra.run.smithi162.stdout:4290 punch from 0xf0288c to 0xf07696, (0x4e0a bytes) 2024-09-16T11:14:49.144 INFO:teuthology.orchestra.run.smithi162.stdout:4293 trunc from 0xa74bf16 to 0xedc93a 2024-09-16T11:14:49.162 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:49.157+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:49.263 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:49.257+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:49.268 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.268 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.278 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:49.321 INFO:teuthology.orchestra.run.smithi162.stdout:4294 write 0x299713f thru 0x299c58e (0x5450 bytes) 2024-09-16T11:14:49.343 INFO:teuthology.orchestra.run.smithi162.stdout:4295 trunc from 0x299c58f to 0xc72dffd 2024-09-16T11:14:49.354 INFO:teuthology.orchestra.run.smithi162.stdout:4296 read 0x511ba5a thru 0x512829f (0xc846 bytes) 2024-09-16T11:14:49.354 INFO:teuthology.orchestra.run.smithi162.stdout:4297 write 0x17fb8f3 thru 0x1803b70 (0x827e bytes) 2024-09-16T11:14:49.358 INFO:teuthology.orchestra.run.smithi162.stdout:4298 write 0xbbc0ca1 thru 0xbbcaeff (0xa25f bytes) 2024-09-16T11:14:49.363 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:49.359+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:49.367 INFO:teuthology.orchestra.run.smithi162.stdout:4301 punch from 0x35e838e to 0x35f3e6f, (0xbae1 bytes) 2024-09-16T11:14:49.367 INFO:teuthology.orchestra.run.smithi162.stdout:4302 read 0x9d3c867 thru 0x9d3e19b (0x1935 bytes) 2024-09-16T11:14:49.367 INFO:teuthology.orchestra.run.smithi162.stdout:4303 punch from 0x4156bcc to 0x4166081, (0xf4b5 bytes) 2024-09-16T11:14:49.367 INFO:teuthology.orchestra.run.smithi162.stdout:4304 clone 35 order 19 su 262144 sc 2 2024-09-16T11:14:49.428 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.429 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:49.481 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:49.479+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:49.565 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:49.563+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:49.585 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.586 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:49.605 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T11:14:49.613 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:14:49.618 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T11:14:49.667 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:49.661+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:49.693 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.693 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.707 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:49.732 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone34 from 0xc72dffd (overlap 0xedc93a) to 0x4755f7 2024-09-16T11:14:49.739 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:49.768 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:49.761+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:49.817 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.818 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.826 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:49.867 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-16T11:14:49.869 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:49.865+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:49.873 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:49.881 INFO:teuthology.orchestra.run.smithi162.stdout:4305 read 0x9a5de99 thru 0x9a62e99 (0x5001 bytes) 2024-09-16T11:14:49.888 INFO:teuthology.orchestra.run.smithi162.stdout:4306 punch from 0x28987b to 0x297005, (0xd78a bytes) 2024-09-16T11:14:49.889 INFO:teuthology.orchestra.run.smithi162.stdout:4307 read 0x2b8dcc9 thru 0x2b9c6f7 (0xea2f bytes) 2024-09-16T11:14:49.890 INFO:teuthology.orchestra.run.smithi162.stdout:4308 read 0x460f694 thru 0x4611831 (0x219e bytes) 2024-09-16T11:14:49.893 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:49.894 INFO:teuthology.orchestra.run.smithi162.stdout:4309 trunc from 0xc72dffd to 0xb22c6f2 2024-09-16T11:14:49.915 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:49.915 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:49.918 INFO:teuthology.orchestra.run.smithi162.stdout:4311 trunc from 0xb22c6f2 to 0x9725e17 2024-09-16T11:14:49.934 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:49.941 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:49.941 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:49.942 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:49.942 INFO:teuthology.orchestra.run.smithi162.stdout:4313 write 0x30f24a3 thru 0x30f2d87 (0x8e5 bytes) 2024-09-16T11:14:49.949 INFO:teuthology.orchestra.run.smithi162.stdout:4314 read 0x12d83b6 thru 0x12dc05e (0x3ca9 bytes) 2024-09-16T11:14:49.950 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:49.950 INFO:teuthology.orchestra.run.smithi162.stdout:4318 read 0x7b214b thru 0x7b7022 (0x4ed8 bytes) 2024-09-16T11:14:49.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:49.952 INFO:teuthology.orchestra.run.smithi162.stdout:4322 write 0x76ddc1 thru 0x7760a0 (0x82e0 bytes) 2024-09-16T11:14:49.956 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:49.962 INFO:teuthology.orchestra.run.smithi162.stdout:4323 read 0x3c1ef3e thru 0x3c27616 (0x86d9 bytes) 2024-09-16T11:14:49.964 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:49.969 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:49.961+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:49.977 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:49.979 INFO:teuthology.orchestra.run.smithi162.stdout:4327 writesame 0xa524ba0 thru 0xa531096 data_size 0x1(0xc4f7 bytes) 2024-09-16T11:14:49.979 INFO:teuthology.orchestra.run.smithi162.stdout:4328 read 0x55f1566 thru 0x55f5df4 (0x488f bytes) 2024-09-16T11:14:49.979 INFO:teuthology.orchestra.run.smithi162.stdout:4329 write 0x79f729e thru 0x79f72b1 (0x14 bytes) 2024-09-16T11:14:49.986 INFO:teuthology.orchestra.run.smithi162.stdout:4330 write 0x8fd84fb thru 0x8fdee1b (0x6921 bytes) 2024-09-16T11:14:49.991 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:49.991 INFO:teuthology.orchestra.run.smithi162.stdout:4333 read 0x2082f3d thru 0x208fc97 (0xcd5b bytes) 2024-09-16T11:14:49.999 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:50.007 INFO:teuthology.orchestra.run.smithi162.stdout:4334 write 0xa569b5a thru 0xa56d772 (0x3c19 bytes) 2024-09-16T11:14:50.012 INFO:teuthology.orchestra.run.smithi162.stdout:4335 writesame 0x756be99 thru 0x7572d37 data_size 0x1(0x6e9f bytes) 2024-09-16T11:14:50.025 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:50.027 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:50.027 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:50.027 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:50.028 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:50.033 INFO:teuthology.orchestra.run.smithi162.stdout:4336 read 0x5ac58d6 thru 0x5acabfe (0x5329 bytes) 2024-09-16T11:14:50.034 INFO:teuthology.orchestra.run.smithi162.stdout:4338 write 0xa3bfb6b thru 0xa3c1fb2 (0x2448 bytes) 2024-09-16T11:14:50.042 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:50.042 INFO:teuthology.orchestra.run.smithi162.stdout:4340 read 0x5752bc0 thru 0x575dba5 (0xafe6 bytes) 2024-09-16T11:14:50.043 INFO:teuthology.orchestra.run.smithi162.stdout:4342 writesame 0x2ef5543 thru 0x2ef8b1a data_size 0x4(0x35d8 bytes) 2024-09-16T11:14:50.053 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:50.053 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:50.053 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:50.053 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:50.053 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:50.057 INFO:teuthology.orchestra.run.smithi162.stdout:4343 trunc from 0xa56d773 to 0x7beb7ce 2024-09-16T11:14:50.062 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:50.079 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:50.079 INFO:teuthology.orchestra.run.smithi162.stdout:4346 writesame 0x21c61bb thru 0x21cad87 data_size 0x1(0x4bcd bytes) 2024-09-16T11:14:50.100 INFO:teuthology.orchestra.run.smithi162.stdout:4347 read 0x4b00280 thru 0x4b0f4b2 (0xf233 bytes) 2024-09-16T11:14:50.102 INFO:teuthology.orchestra.run.smithi162.stdout:4349 write 0x2edbcb2 thru 0x2ee796d (0xbcbc bytes) 2024-09-16T11:14:50.106 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:50.101+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:50.108 INFO:teuthology.orchestra.run.smithi162.stdout:4350 write 0x4652d86 thru 0x465a4ce (0x7749 bytes) 2024-09-16T11:14:50.129 INFO:teuthology.orchestra.run.smithi162.stdout:4351 writesame 0x82effc7 thru 0x82f275e data_size 0x1c(0x2798 bytes) 2024-09-16T11:14:50.129 INFO:teuthology.orchestra.run.smithi162.stdout:4352 read 0x5367b40 thru 0x5373364 (0xb825 bytes) 2024-09-16T11:14:50.129 INFO:teuthology.orchestra.run.smithi162.stdout:4353 punch from 0x5a78159 to 0x5a823b3, (0xa25a bytes) 2024-09-16T11:14:50.130 INFO:teuthology.orchestra.run.smithi162.stdout:4354 read 0x7c20951 thru 0x7c2ebc3 (0xe273 bytes) 2024-09-16T11:14:50.130 INFO:teuthology.orchestra.run.smithi162.stdout:4355 write 0x80c9e32 thru 0x80d8d71 (0xef40 bytes) 2024-09-16T11:14:50.138 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:50.165 INFO:teuthology.orchestra.run.smithi162.stdout:4357 write 0xee565d2 thru 0xee58e83 (0x28b2 bytes) 2024-09-16T11:14:50.171 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:50.167+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:50.171 INFO:teuthology.orchestra.run.smithi162.stdout:4358 read 0xc1eef3 thru 0xc2d8d1 (0xe9df bytes) 2024-09-16T11:14:50.173 INFO:teuthology.orchestra.run.smithi162.stdout:4359 write 0x6916ae thru 0x697f1f (0x6872 bytes) 2024-09-16T11:14:50.208 INFO:teuthology.orchestra.run.smithi162.stdout:4360 trunc from 0xee58e84 to 0x294f88b 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135285060000001, 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:50.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:50.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.219 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135265694000001, 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:50.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:14:50.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:14:50.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:14:50.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135224932, 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.228 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:50.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:14:50.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135194133999999, 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:50.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:50.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:14:50.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135175545999999, 2024-09-16T11:14:50.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:50.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.244 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135157494000001, 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:50.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:50.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135137567000001, 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:14:50.250 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:50.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:50.255 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.135118727999998, 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:50.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:14:50.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.134926779000001, 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:50.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:50.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:50.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:14:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:14:50.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:14:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:14:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.134907499000001, 2024-09-16T11:14:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:14:50.271 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:50.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:50.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:50.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:14:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:14:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:14:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:14:50.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131437234, 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:14:50.282 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:14:50.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:14:50.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131415930999999, 2024-09-16T11:14:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:14:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.289 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.290 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:50.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:14:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:14:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:14:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:14:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:14:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:14:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131405860000001, 2024-09-16T11:14:50.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:14:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.302 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:50.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:50.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:14:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:14:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:14:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:14:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:14:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:14:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131377557, 2024-09-16T11:14:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:14:50.313 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.314 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:14:50.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:14:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:14:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:14:50.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:14:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:14:50.324 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.325 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131363627999999, 2024-09-16T11:14:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:14:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:50.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:50.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:50.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:14:50.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:14:50.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:14:50.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:14:50.333 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:50.334 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131354864999999, 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:50.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:50.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131331046, 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:14:50.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131299924, 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:14:50.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:14:50.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131252185000001, 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.131230077000001, 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.356 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:14:50.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.361 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.361 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:50.362 INFO:teuthology.orchestra.run.smithi162.stdout:4362 trunc from 0x294f88b to 0xc672eb9 2024-09-16T11:14:50.362 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:50.362 INFO:teuthology.orchestra.run.smithi162.stdout:4364 write 0xed23e5c thru 0xed2c6c4 (0x8869 bytes) 2024-09-16T11:14:50.363 INFO:teuthology.orchestra.run.smithi162.stdout:4365 write 0xd07fdb3 thru 0xd0851cf (0x541d bytes) 2024-09-16T11:14:50.363 INFO:teuthology.orchestra.run.smithi162.stdout:4367 clone 36 order 21 su 8192 sc 10 2024-09-16T11:14:50.363 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:50.359+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:50.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:50.463 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:50.459+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:50.472 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:50.472 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:50.472 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:50.472 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:50.482 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:50.564 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:50.557+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:50.585 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:50.585 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:50.585 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:50.585 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:50.585 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:50.594 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:50.620 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T11:14:50.634 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T11:14:50.677 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:50.675+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.834741557000001, 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:14:50.701 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.705 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666769907999999, 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.706 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:14:50.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666758346999998, 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:14:50.710 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:50.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:14:50.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:14:50.714 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666745145, 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:14:50.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.719 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666732705000001, 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.720 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666719194000002, 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.724 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.728 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666708278999998, 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:14:50.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:14:50.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666682360999999, 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.735 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:14:50.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666666907, 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:14:50.739 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.743 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666657020999999, 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.744 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:14:50.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666647257999998, 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.749 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:14:50.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.753 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666637080999998, 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:50.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:14:50.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666620598999998, 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:14:50.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:50.762 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666605902000001, 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:14:50.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:14:50.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666587458999999, 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.768 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:14:50.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.66657051, 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.772 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:14:50.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:14:50.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666551296999998, 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:14:50.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666534081000002, 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:14:50.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:14:50.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666512826999998, 2024-09-16T11:14:50.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:14:50.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.666492859000002, 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.791 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:14:50.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:50.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:50.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:50.795 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:50.796 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone35 from 0xed2c6c5 (overlap 0x294f88b) to 0x4218da 2024-09-16T11:14:50.797 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:50.797 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:50.789+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:50.897 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:50.893+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:50.910 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:50.910 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:50.910 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:50.910 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:50.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:50.998 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:50.995+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:51.022 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:51.022 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:51.022 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:51.023 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:51.023 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:51.031 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:51.047 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-16T11:14:51.077 INFO:teuthology.orchestra.run.smithi162.stdout:4368 write 0x585b4cc thru 0x5868d07 (0xd83c bytes) 2024-09-16T11:14:51.098 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:51.095+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:51.103 INFO:teuthology.orchestra.run.smithi162.stdout:4369 write 0x9d4a64d thru 0x9d4be29 (0x17dd bytes) 2024-09-16T11:14:51.108 INFO:teuthology.orchestra.run.smithi162.stdout:4370 writesame 0x298d1ed thru 0x299ccaa data_size 0x83(0xfabe bytes) 2024-09-16T11:14:51.130 INFO:teuthology.orchestra.run.smithi162.stdout:4372 read 0x6a1692 thru 0x6aa63c (0x8fab bytes) 2024-09-16T11:14:51.139 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.950413939000001, 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:14:51.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:14:51.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.144 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.946055412, 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:14:51.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:14:51.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.942416882000003, 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.149 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:14:51.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.153 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.942398095999998, 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:14:51.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.942380772999996, 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.158 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:14:51.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.162 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.942363303999997, 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:14:51.166 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938522845000001, 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:14:51.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.172 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.172 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938512729999999, 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:51.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938502849000002, 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:14:51.177 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:14:51.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938493672999996, 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:14:51.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:14:51.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938477605999999, 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.186 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:14:51.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.190 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938461211000003, 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:51.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:14:51.194 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938451637, 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:14:51.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:14:51.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.938442101, 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:14:51.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.202 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.934385444, 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:14:51.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.206 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.934376061999998, 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.207 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:14:51.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.210 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.934367404, 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:51.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:51.215 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.934358170000003, 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:51.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:14:51.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.240605217999999, 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:14:51.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:14:51.223 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.554769963999998, 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:51.224 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:14:51.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:14:51.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.230 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:51.231 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.231 INFO:teuthology.orchestra.run.smithi162.stdout:4377 write 0xdf7beb5 thru 0xdf8adb7 (0xef03 bytes) 2024-09-16T11:14:51.232 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:51.232 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:51.225+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:51.241 INFO:teuthology.orchestra.run.smithi162.stdout:4382 trunc from 0xed2c6c5 to 0x1820c7b 2024-09-16T11:14:51.332 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:51.325+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:51.347 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:51.348 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:51.357 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:51.404 INFO:teuthology.orchestra.run.smithi162.stdout:4384 read 0x1624d66 thru 0x1631817 (0xcab2 bytes) 2024-09-16T11:14:51.432 INFO:teuthology.orchestra.run.smithi162.stdout:4385 write 0xc502948 thru 0xc505328 (0x29e1 bytes) 2024-09-16T11:14:51.433 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:51.431+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:51.440 INFO:teuthology.orchestra.run.smithi162.stdout:4386 writesame 0x2d9ea3c thru 0x2da061e data_size 0xb(0x1be3 bytes) 2024-09-16T11:14:51.447 INFO:teuthology.orchestra.run.smithi162.stdout:4387 punch from 0x367a520 to 0x3683c9a, (0x977a bytes) 2024-09-16T11:14:51.447 INFO:teuthology.orchestra.run.smithi162.stdout:4388 read 0x208ec15 thru 0x2099ff5 (0xb3e1 bytes) 2024-09-16T11:14:51.448 INFO:teuthology.orchestra.run.smithi162.stdout:4389 punch from 0xb318ccb to 0xb31deb6, (0x51eb bytes) 2024-09-16T11:14:51.448 INFO:teuthology.orchestra.run.smithi162.stdout:4391 punch from 0xae436f5 to 0xae4d065, (0x9970 bytes) 2024-09-16T11:14:51.448 INFO:teuthology.orchestra.run.smithi162.stdout:4393 read 0xbf15539 thru 0xbf1bf41 (0x6a09 bytes) 2024-09-16T11:14:51.449 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.449 INFO:teuthology.orchestra.run.smithi162.stdout:4395 punch from 0x773f1a0 to 0x773f361, (0x1c1 bytes) 2024-09-16T11:14:51.449 INFO:teuthology.orchestra.run.smithi162.stdout:4396 read 0x41ba552 thru 0x41bf9af (0x545e bytes) 2024-09-16T11:14:51.449 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.449 INFO:teuthology.orchestra.run.smithi162.stdout:4401 write 0x51cd5d1 thru 0x51d00a6 (0x2ad6 bytes) 2024-09-16T11:14:51.451 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:51.451 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:51.451 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:51.451 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:51.451 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:51.461 INFO:teuthology.orchestra.run.smithi162.stdout:4402 punch from 0x86561db to 0x865e2b7, (0x80dc bytes) 2024-09-16T11:14:51.461 INFO:teuthology.orchestra.run.smithi162.stdout:4403 read 0x38a4fd9 thru 0x38a5dc8 (0xdf0 bytes) 2024-09-16T11:14:51.461 INFO:teuthology.orchestra.run.smithi162.stdout:4406 write 0xacf26a5 thru 0xacfcc37 (0xa593 bytes) 2024-09-16T11:14:51.462 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:51.475 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.479 INFO:teuthology.orchestra.run.smithi162.stdout:4409 trunc from 0xc505329 to 0xe9263c3 2024-09-16T11:14:51.488 INFO:teuthology.orchestra.run.smithi162.stdout:4410 read 0x55543de thru 0x5562388 (0xdfab bytes) 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:4413 punch from 0x4ebea8c to 0x4ebefa5, (0x519 bytes) 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:4414 read 0x552e414 thru 0x553a7a0 (0xc38d bytes) 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:4415 read 0x4c34e46 thru 0x4c3b157 (0x6312 bytes) 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:4416 punch from 0x4b5f147 to 0x4b679ec, (0x88a5 bytes) 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:4417 read 0xd88c814 thru 0xd89c0f6 (0xf8e3 bytes) 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.489 INFO:teuthology.orchestra.run.smithi162.stdout:4420 write 0x5d37afb thru 0x5d43d8e (0xc294 bytes) 2024-09-16T11:14:51.502 INFO:teuthology.orchestra.run.smithi162.stdout:4422 read 0xc4e16a thru 0xc53ee5 (0x5d7c bytes) 2024-09-16T11:14:51.506 INFO:teuthology.orchestra.run.smithi162.stdout:4423 write 0xfb9619 thru 0xfc20b0 (0x8a98 bytes) 2024-09-16T11:14:51.535 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:51.531+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:51.573 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:51.573 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:51.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:51.573 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:51.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.573 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.79344947200001, 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:51.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:51.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:51.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:51.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.583 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.405523772, 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:51.584 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:51.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:51.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:51.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.405372442, 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:51.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:51.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.405320956, 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.600 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:51.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:51.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.40525080800001, 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.607 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:51.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:51.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.405078338, 2024-09-16T11:14:51.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:51.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:51.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:51.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:51.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:51.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:51.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:51.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:51.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:51.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:51.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:51.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:51.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:51.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.624 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.624 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.624 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:51.624 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.40502001599999, 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.625 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:51.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.626 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:51.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:51.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:51.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:51.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:51.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:51.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:51.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:51.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:51.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:51.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:51.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:51.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.635 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.404867896, 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:51.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:51.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:51.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.642 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.643 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:51.643 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.404689377, 2024-09-16T11:14:51.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:51.643 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.643 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.644 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:51.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.644 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:51.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:51.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:51.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.651 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 121.40465640399999, 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:51.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:51.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:51.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.659 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.852468679, 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.660 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:51.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:51.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:51.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:51.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:51.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.672 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.852339882, 2024-09-16T11:14:51.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:51.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.675 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:51.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:51.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:51.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:51.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.681 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.682 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.682 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:51.682 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.852281537, 2024-09-16T11:14:51.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:51.682 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.683 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.683 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:51.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:51.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:51.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.852227148, 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.690 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:51.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:51.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.852092841, 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:51.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.070937472, 2024-09-16T11:14:51.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:51.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:51.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.07085016000001, 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.709 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:51.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:51.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 120.070612032, 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:51.716 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:51.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:51.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 49.390232023000003, 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:51.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:51.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 34.682039586999998, 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:51.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:14:51.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:51.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:51.730 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:51.731 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:14:51.733 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:51.725+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:51.733 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:4425 write 0xb8c4e76 thru 0xb8ce43b (0x95c6 bytes) 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:4427 write 0xc344c2c thru 0xc3539e0 (0xedb5 bytes) 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:4428 read 0x353f2db thru 0x3541e37 (0x2b5d bytes) 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:4429 writesame 0x1673e1 thru 0x1681c8 data_size 0x14(0xde8 bytes) 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:4432 write 0xe0e46a0 thru 0xe0f1b24 (0xd485 bytes) 2024-09-16T11:14:51.734 INFO:teuthology.orchestra.run.smithi162.stdout:4433 writesame 0xe8eb296 thru 0xe8f0996 data_size 0x1(0x5701 bytes) 2024-09-16T11:14:51.735 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T11:14:51.735 INFO:teuthology.orchestra.run.smithi162.stdout:4436 punch from 0xc4eedba to 0xc4f93ce, (0xa614 bytes) 2024-09-16T11:14:51.735 INFO:teuthology.orchestra.run.smithi162.stdout:4437 read 0xa243637 thru 0xa24b820 (0x81ea bytes) 2024-09-16T11:14:51.735 INFO:teuthology.orchestra.run.smithi162.stdout:4438 write 0x725aa8d thru 0x72604fa (0x5a6e bytes) 2024-09-16T11:14:51.736 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T11:14:51.737 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:51.737 INFO:teuthology.orchestra.run.smithi162.stdout:4441 read 0x5cd0924 thru 0x5cd2746 (0x1e23 bytes) 2024-09-16T11:14:51.737 INFO:teuthology.orchestra.run.smithi162.stdout:4442 write 0xe73ac68 thru 0xe743cf0 (0x9089 bytes) 2024-09-16T11:14:51.737 INFO:teuthology.orchestra.run.smithi162.stdout:4443 read 0xda79a10 thru 0xda7ace6 (0x12d7 bytes) 2024-09-16T11:14:51.738 INFO:teuthology.orchestra.run.smithi162.stdout:4444 flatten 2024-09-16T11:14:51.738 INFO:teuthology.orchestra.run.smithi162.stdout:4445 write 0x3f71f1c thru 0x3f7edc2 (0xcea7 bytes) 2024-09-16T11:14:51.738 INFO:teuthology.orchestra.run.smithi162.stdout:4446 write 0xd08cf72 thru 0xd08f72c (0x27bb bytes) 2024-09-16T11:14:51.738 INFO:teuthology.orchestra.run.smithi162.stdout:4447 clone 37 order 18 su 2048 sc 3 2024-09-16T11:14:51.834 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:51.829+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:51.890 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:51.890 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:51.890 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:51.890 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:51.902 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:51.934 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:51.929+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:51.994 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:51.994 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:51.995 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:51.995 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:51.995 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:52.004 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:52.035 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:52.029+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:52.041 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:52.112 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:52.112 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:52.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:52.112 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:52.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.933670724999999, 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.933592981999993, 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.117 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:52.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:52.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.931226967000001, 2024-09-16T11:14:52.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.125 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.931207451000006, 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:52.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.129 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.930069947000007, 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:52.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.930054048000002, 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.134 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:52.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.930029794000006, 2024-09-16T11:14:52.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.930014534999998, 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:52.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:52.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:52.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.929999215999999, 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:52.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927976811999997, 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927962313999998, 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:52.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:52.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927947981000003, 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:52.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927933924000001, 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:52.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:52.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:52.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927920068000006, 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:52.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927906192999998, 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927892426, 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:52.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927844917000002, 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.181 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.927559099000007, 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:52.185 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:52.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:52.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.834895282999994, 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:52.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:52.194 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.834746100999993, 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:52.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:52.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:52.200 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:52.202 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:52.202 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:52.203 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:52.197+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:52.224 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:52.243 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:52.264 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:52.287 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:52.302 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:52.302 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:52.303 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:52.297+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:52.310 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:52.311 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:52.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:52.355 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:52.356 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:52.357 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:52.407 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:52.407 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:52.408 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:52.403+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:52.416 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:52.505 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:52.497+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:52.554 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:52.554 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:52.563 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:52.606 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:52.601+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:52.636 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T11:14:52.641 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:14:52.650 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T11:14:52.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:14:52.670 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:52.670 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:52.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:52.707 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:52.701+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:52.776 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:52.776 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:52.787 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:52.808 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:52.801+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:52.898 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:52.898 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:52.908 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:52.909 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:52.901+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:52.961 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-16T11:14:52.966 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:14:52.968 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:14:52.984 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-16T11:14:52.626605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1693685330,"num_objects":2410,"num_object_clones":495,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":631,"num_read":194560,"num_read_kb":1301418,"num_write":20157,"num_write_kb":717530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1519,"num_bytes_recovered":218241032,"num_keys_recovered":271,"num_objects_omap":39,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14973,"ondisk_log_size":14973,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":3109208,"kb_used_data":965240,"kb_used_omap":11,"kb_used_meta":2143796,"kb_avail":657493672,"statfs":{"total":676457349120,"available":673273520128,"internally_reserved":0,"allocated":988405760,"data_stored":1068309681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2195247464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":185401918,"num_objects":222,"num_object_clones":40,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":90,"num_read":14725,"num_read_kb":137706,"num_write":961,"num_write_kb":64263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035720"},"pg_stats":[{"pgid":"4.1f","version":"108'284","reported_seq":5474,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.630784+0000","last_change":"2024-09-16T11:13:15.136003+0000","last_active":"2024-09-16T11:14:51.630784+0000","last_peered":"2024-09-16T11:14:51.630784+0000","last_clean":"2024-09-16T11:14:51.630784+0000","last_became_active":"2024-09-16T11:13:13.110861+0000","last_became_peered":"2024-09-16T11:13:13.110861+0000","last_unstale":"2024-09-16T11:14:51.630784+0000","last_undegraded":"2024-09-16T11:14:51.630784+0000","last_fullsized":"2024-09-16T11:14:51.630784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:49.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002295123,"stat_sum":{"num_bytes":29854866,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":22,"num_read":6181,"num_read_kb":29712,"num_write":270,"num_write_kb":10157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"108'581","reported_seq":10209,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.652609+0000","last_change":"2024-09-16T11:13:15.329907+0000","last_active":"2024-09-16T11:14:51.652609+0000","last_peered":"2024-09-16T11:14:51.652609+0000","last_clean":"2024-09-16T11:14:51.652609+0000","last_became_active":"2024-09-16T11:13:13.129077+0000","last_became_peered":"2024-09-16T11:13:13.129077+0000","last_unstale":"2024-09-16T11:14:51.652609+0000","last_undegraded":"2024-09-16T11:14:51.652609+0000","last_fullsized":"2024-09-16T11:14:51.652609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:56.981990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050881799999999997,"stat_sum":{"num_bytes":64992645,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":17,"num_read":12166,"num_read_kb":90256,"num_write":781,"num_write_kb":41396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"108'351","reported_seq":4177,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.687197+0000","last_change":"2024-09-16T11:13:15.119814+0000","last_active":"2024-09-16T11:14:51.687197+0000","last_peered":"2024-09-16T11:14:51.687197+0000","last_clean":"2024-09-16T11:14:51.687197+0000","last_became_active":"2024-09-16T11:13:13.110835+0000","last_became_peered":"2024-09-16T11:13:13.110835+0000","last_unstale":"2024-09-16T11:14:51.687197+0000","last_undegraded":"2024-09-16T11:14:51.687197+0000","last_fullsized":"2024-09-16T11:14:51.687197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:25.687407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063997100000000003,"stat_sum":{"num_bytes":45525761,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":16,"num_read":4477,"num_read_kb":25852,"num_write":485,"num_write_kb":14109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"107'804","reported_seq":5107,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.534836+0000","last_change":"2024-09-16T11:14:24.176887+0000","last_active":"2024-09-16T11:14:51.534836+0000","last_peered":"2024-09-16T11:14:51.534836+0000","last_clean":"2024-09-16T11:14:51.534836+0000","last_became_active":"2024-09-16T11:14:24.105273+0000","last_became_peered":"2024-09-16T11:14:24.105273+0000","last_unstale":"2024-09-16T11:14:51.534836+0000","last_undegraded":"2024-09-16T11:14:51.534836+0000","last_fullsized":"2024-09-16T11:14:51.534836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:10.103253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038046360000000001,"stat_sum":{"num_bytes":38541162,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":23,"num_read":8028,"num_read_kb":29663,"num_write":1203,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2431957,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"107'349","reported_seq":4462,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.566038+0000","last_change":"2024-09-16T11:14:45.568220+0000","last_active":"2024-09-16T11:14:51.566038+0000","last_peered":"2024-09-16T11:14:51.566038+0000","last_clean":"2024-09-16T11:14:51.566038+0000","last_became_active":"2024-09-16T11:14:45.567373+0000","last_became_peered":"2024-09-16T11:14:45.567373+0000","last_unstale":"2024-09-16T11:14:51.566038+0000","last_undegraded":"2024-09-16T11:14:51.566038+0000","last_fullsized":"2024-09-16T11:14:51.566038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.613053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029419009999999998,"stat_sum":{"num_bytes":19443080,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":15,"num_read":4587,"num_read_kb":15973,"num_write":483,"num_write_kb":13985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":8560528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"108'428","reported_seq":3688,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.677326+0000","last_change":"2024-09-16T11:14:34.444229+0000","last_active":"2024-09-16T11:14:51.677326+0000","last_peered":"2024-09-16T11:14:51.677326+0000","last_clean":"2024-09-16T11:14:51.677326+0000","last_became_active":"2024-09-16T11:14:34.442783+0000","last_became_peered":"2024-09-16T11:14:34.442783+0000","last_unstale":"2024-09-16T11:14:51.677326+0000","last_undegraded":"2024-09-16T11:14:51.677326+0000","last_fullsized":"2024-09-16T11:14:51.677326+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:44:14.832997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024433810000000001,"stat_sum":{"num_bytes":44160458,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":16,"num_read":4352,"num_read_kb":31078,"num_write":559,"num_write_kb":20751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":17721559,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"107'390","reported_seq":2401,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.546450+0000","last_change":"2024-09-16T11:14:23.090655+0000","last_active":"2024-09-16T11:14:51.546450+0000","last_peered":"2024-09-16T11:14:51.546450+0000","last_clean":"2024-09-16T11:14:51.546450+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:14:51.546450+0000","last_undegraded":"2024-09-16T11:14:51.546450+0000","last_fullsized":"2024-09-16T11:14:51.546450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072097299999999997,"stat_sum":{"num_bytes":53879978,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":25,"num_read":3163,"num_read_kb":19399,"num_write":559,"num_write_kb":17721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"107'535","reported_seq":4689,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.124537+0000","last_change":"2024-09-16T11:14:34.443921+0000","last_active":"2024-09-16T11:14:51.124537+0000","last_peered":"2024-09-16T11:14:51.124537+0000","last_clean":"2024-09-16T11:14:51.124537+0000","last_became_active":"2024-09-16T11:14:34.443009+0000","last_became_peered":"2024-09-16T11:14:34.443009+0000","last_unstale":"2024-09-16T11:14:51.124537+0000","last_undegraded":"2024-09-16T11:14:51.124537+0000","last_fullsized":"2024-09-16T11:14:51.124537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:23.629543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055416799999999996,"stat_sum":{"num_bytes":82359608,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":16,"num_read":6231,"num_read_kb":34918,"num_write":629,"num_write_kb":27883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":2375829,"num_keys_recovered":32,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"107'1411","reported_seq":6010,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.599688+0000","last_change":"2024-09-16T11:13:15.120697+0000","last_active":"2024-09-16T11:14:51.599688+0000","last_peered":"2024-09-16T11:14:51.599688+0000","last_clean":"2024-09-16T11:14:51.599688+0000","last_became_active":"2024-09-16T11:13:13.131852+0000","last_became_peered":"2024-09-16T11:13:13.131852+0000","last_unstale":"2024-09-16T11:14:51.599688+0000","last_undegraded":"2024-09-16T11:14:51.599688+0000","last_fullsized":"2024-09-16T11:14:51.599688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:23.813033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041000900000000002,"stat_sum":{"num_bytes":25509672,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":11625,"num_read_kb":16369,"num_write":2513,"num_write_kb":7973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"108'320","reported_seq":6519,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.636595+0000","last_change":"2024-09-16T11:13:15.330097+0000","last_active":"2024-09-16T11:14:51.636595+0000","last_peered":"2024-09-16T11:14:51.636595+0000","last_clean":"2024-09-16T11:14:51.636595+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:14:51.636595+0000","last_undegraded":"2024-09-16T11:14:51.636595+0000","last_fullsized":"2024-09-16T11:14:51.636595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061357700000000002,"stat_sum":{"num_bytes":59184687,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":21,"num_read":6964,"num_read_kb":49200,"num_write":396,"num_write_kb":15820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"107'312","reported_seq":2904,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.523249+0000","last_change":"2024-09-16T11:13:15.110275+0000","last_active":"2024-09-16T11:14:51.523249+0000","last_peered":"2024-09-16T11:14:51.523249+0000","last_clean":"2024-09-16T11:14:51.523249+0000","last_became_active":"2024-09-16T11:13:13.127081+0000","last_became_peered":"2024-09-16T11:13:13.127081+0000","last_unstale":"2024-09-16T11:14:51.523249+0000","last_undegraded":"2024-09-16T11:14:51.523249+0000","last_fullsized":"2024-09-16T11:14:51.523249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:29:58.185747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065955699999999996,"stat_sum":{"num_bytes":46667583,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":28,"num_read":3026,"num_read_kb":15217,"num_write":288,"num_write_kb":25173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"108'463","reported_seq":5355,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627741+0000","last_change":"2024-09-16T11:13:15.120212+0000","last_active":"2024-09-16T11:14:51.627741+0000","last_peered":"2024-09-16T11:14:51.627741+0000","last_clean":"2024-09-16T11:14:51.627741+0000","last_became_active":"2024-09-16T11:13:13.116260+0000","last_became_peered":"2024-09-16T11:13:13.116260+0000","last_unstale":"2024-09-16T11:14:51.627741+0000","last_undegraded":"2024-09-16T11:14:51.627741+0000","last_fullsized":"2024-09-16T11:14:51.627741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:32.165127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066315600000000003,"stat_sum":{"num_bytes":80787687,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":22,"num_read":6675,"num_read_kb":58352,"num_write":439,"num_write_kb":26774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"107'359","reported_seq":3224,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.808050+0000","last_change":"2024-09-16T11:14:44.550763+0000","last_active":"2024-09-16T11:14:50.808050+0000","last_peered":"2024-09-16T11:14:50.808050+0000","last_clean":"2024-09-16T11:14:50.808050+0000","last_became_active":"2024-09-16T11:14:44.549012+0000","last_became_peered":"2024-09-16T11:14:44.549012+0000","last_unstale":"2024-09-16T11:14:50.808050+0000","last_undegraded":"2024-09-16T11:14:50.808050+0000","last_fullsized":"2024-09-16T11:14:50.808050+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:38.040942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028235669999999999,"stat_sum":{"num_bytes":45646427,"num_objects":82,"num_object_clones":17,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":13,"num_read":3718,"num_read_kb":37178,"num_write":412,"num_write_kb":22646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":33382609,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"108'329","reported_seq":4389,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.621410+0000","last_change":"2024-09-16T11:14:08.511426+0000","last_active":"2024-09-16T11:14:51.621410+0000","last_peered":"2024-09-16T11:14:51.621410+0000","last_clean":"2024-09-16T11:14:51.621410+0000","last_became_active":"2024-09-16T11:14:08.510206+0000","last_became_peered":"2024-09-16T11:14:08.510206+0000","last_unstale":"2024-09-16T11:14:51.621410+0000","last_undegraded":"2024-09-16T11:14:51.621410+0000","last_fullsized":"2024-09-16T11:14:51.621410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:41.370533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145994,"stat_sum":{"num_bytes":83890873,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":16,"num_read":4449,"num_read_kb":51969,"num_write":581,"num_write_kb":22511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"107'490","reported_seq":3657,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627824+0000","last_change":"2024-09-16T11:13:15.119821+0000","last_active":"2024-09-16T11:14:51.627824+0000","last_peered":"2024-09-16T11:14:51.627824+0000","last_clean":"2024-09-16T11:14:51.627824+0000","last_became_active":"2024-09-16T11:13:13.131936+0000","last_became_peered":"2024-09-16T11:13:13.131936+0000","last_unstale":"2024-09-16T11:14:51.627824+0000","last_undegraded":"2024-09-16T11:14:51.627824+0000","last_fullsized":"2024-09-16T11:14:51.627824+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:24.053000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059135899999999996,"stat_sum":{"num_bytes":100058172,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":25,"num_read":4918,"num_read_kb":87602,"num_write":565,"num_write_kb":44087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"107'445","reported_seq":3918,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.545980+0000","last_change":"2024-09-16T11:14:45.570341+0000","last_active":"2024-09-16T11:14:51.545980+0000","last_peered":"2024-09-16T11:14:51.545980+0000","last_clean":"2024-09-16T11:14:51.545980+0000","last_became_active":"2024-09-16T11:14:45.569559+0000","last_became_peered":"2024-09-16T11:14:45.569559+0000","last_unstale":"2024-09-16T11:14:51.545980+0000","last_undegraded":"2024-09-16T11:14:51.545980+0000","last_fullsized":"2024-09-16T11:14:51.545980+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:51.205615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050522900000000005,"stat_sum":{"num_bytes":49539443,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":22,"num_read":4691,"num_read_kb":29800,"num_write":636,"num_write_kb":22416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":18357825,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.619688+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:14:50.619688+0000","last_peered":"2024-09-16T11:14:50.619688+0000","last_clean":"2024-09-16T11:14:50.619688+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:14:50.619688+0000","last_undegraded":"2024-09-16T11:14:50.619688+0000","last_fullsized":"2024-09-16T11:14:50.619688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"108'681","reported_seq":3716,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.658163+0000","last_change":"2024-09-16T11:14:33.441244+0000","last_active":"2024-09-16T11:14:51.658163+0000","last_peered":"2024-09-16T11:14:51.658163+0000","last_clean":"2024-09-16T11:14:51.658163+0000","last_became_active":"2024-09-16T11:14:33.439983+0000","last_became_peered":"2024-09-16T11:14:33.439983+0000","last_unstale":"2024-09-16T11:14:51.658163+0000","last_undegraded":"2024-09-16T11:14:51.658163+0000","last_fullsized":"2024-09-16T11:14:51.658163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:04.864478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071029199999999995,"stat_sum":{"num_bytes":35174535,"num_objects":81,"num_object_clones":16,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":27,"num_read":5882,"num_read_kb":16040,"num_write":882,"num_write_kb":7466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":3634035,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":102,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.614956+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:14:50.614956+0000","last_peered":"2024-09-16T11:14:50.614956+0000","last_clean":"2024-09-16T11:14:50.614956+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:14:50.614956+0000","last_undegraded":"2024-09-16T11:14:50.614956+0000","last_fullsized":"2024-09-16T11:14:50.614956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"108'302","reported_seq":4164,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627521+0000","last_change":"2024-09-16T11:13:15.119575+0000","last_active":"2024-09-16T11:14:51.627521+0000","last_peered":"2024-09-16T11:14:51.627521+0000","last_clean":"2024-09-16T11:14:51.627521+0000","last_became_active":"2024-09-16T11:13:13.131951+0000","last_became_peered":"2024-09-16T11:13:13.131951+0000","last_unstale":"2024-09-16T11:14:51.627521+0000","last_undegraded":"2024-09-16T11:14:51.627521+0000","last_fullsized":"2024-09-16T11:14:51.627521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:26.039221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051524000000000003,"stat_sum":{"num_bytes":47876326,"num_objects":83,"num_object_clones":18,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":27,"num_read":4182,"num_read_kb":35531,"num_write":324,"num_write_kb":11293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.611845+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:14:50.611845+0000","last_peered":"2024-09-16T11:14:50.611845+0000","last_clean":"2024-09-16T11:14:50.611845+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:14:50.611845+0000","last_undegraded":"2024-09-16T11:14:50.611845+0000","last_fullsized":"2024-09-16T11:14:50.611845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"108'763","reported_seq":10585,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.690315+0000","last_change":"2024-09-16T11:13:15.330268+0000","last_active":"2024-09-16T11:14:51.690315+0000","last_peered":"2024-09-16T11:14:51.690315+0000","last_clean":"2024-09-16T11:14:51.690315+0000","last_became_active":"2024-09-16T11:13:13.112956+0000","last_became_peered":"2024-09-16T11:13:13.112956+0000","last_unstale":"2024-09-16T11:14:51.690315+0000","last_undegraded":"2024-09-16T11:14:51.690315+0000","last_fullsized":"2024-09-16T11:14:51.690315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:37.422681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674599999999998,"stat_sum":{"num_bytes":44856470,"num_objects":91,"num_object_clones":20,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":24,"num_read":12540,"num_read_kb":30045,"num_write":1082,"num_write_kb":20109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.708887+0000","last_change":"2024-09-16T11:13:52.391053+0000","last_active":"2024-09-16T11:14:50.708887+0000","last_peered":"2024-09-16T11:14:50.708887+0000","last_clean":"2024-09-16T11:14:50.708887+0000","last_became_active":"2024-09-16T11:13:52.390767+0000","last_became_peered":"2024-09-16T11:13:52.390767+0000","last_unstale":"2024-09-16T11:14:50.708887+0000","last_undegraded":"2024-09-16T11:14:50.708887+0000","last_fullsized":"2024-09-16T11:14:50.708887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:14.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"107'206","reported_seq":6537,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.446845+0000","last_change":"2024-09-16T11:13:15.112573+0000","last_active":"2024-09-16T11:14:51.446845+0000","last_peered":"2024-09-16T11:14:51.446845+0000","last_clean":"2024-09-16T11:14:51.446845+0000","last_became_active":"2024-09-16T11:13:13.113469+0000","last_became_peered":"2024-09-16T11:13:13.113469+0000","last_unstale":"2024-09-16T11:14:51.446845+0000","last_undegraded":"2024-09-16T11:14:51.446845+0000","last_fullsized":"2024-09-16T11:14:51.446845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:44.833246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815220000000001,"stat_sum":{"num_bytes":24016162,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":19,"num_read":6335,"num_read_kb":26900,"num_write":197,"num_write_kb":9360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.708965+0000","last_change":"2024-09-16T11:14:30.430965+0000","last_active":"2024-09-16T11:14:50.708965+0000","last_peered":"2024-09-16T11:14:50.708965+0000","last_clean":"2024-09-16T11:14:50.708965+0000","last_became_active":"2024-09-16T11:14:30.430629+0000","last_became_peered":"2024-09-16T11:14:30.430629+0000","last_unstale":"2024-09-16T11:14:50.708965+0000","last_undegraded":"2024-09-16T11:14:50.708965+0000","last_fullsized":"2024-09-16T11:14:50.708965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:44.285723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":104,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.622947+0000","last_change":"2024-09-16T11:12:56.834191+0000","last_active":"2024-09-16T11:14:50.622947+0000","last_peered":"2024-09-16T11:14:50.622947+0000","last_clean":"2024-09-16T11:14:50.622947+0000","last_became_active":"2024-09-16T11:12:54.819534+0000","last_became_peered":"2024-09-16T11:12:54.819534+0000","last_unstale":"2024-09-16T11:14:50.622947+0000","last_undegraded":"2024-09-16T11:14:50.622947+0000","last_fullsized":"2024-09-16T11:14:50.622947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:47.418906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"108'474","reported_seq":5915,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.653059+0000","last_change":"2024-09-16T11:14:23.105967+0000","last_active":"2024-09-16T11:14:51.653059+0000","last_peered":"2024-09-16T11:14:51.653059+0000","last_clean":"2024-09-16T11:14:51.653059+0000","last_became_active":"2024-09-16T11:14:23.092519+0000","last_became_peered":"2024-09-16T11:14:23.092519+0000","last_unstale":"2024-09-16T11:14:51.653059+0000","last_undegraded":"2024-09-16T11:14:51.653059+0000","last_fullsized":"2024-09-16T11:14:51.653059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:58.815065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001093002,"stat_sum":{"num_bytes":64165959,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":12,"num_read":7114,"num_read_kb":58615,"num_write":850,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":14007837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.643500+0000","last_change":"2024-09-16T11:14:20.060782+0000","last_active":"2024-09-16T11:14:50.643500+0000","last_peered":"2024-09-16T11:14:50.643500+0000","last_clean":"2024-09-16T11:14:50.643500+0000","last_became_active":"2024-09-16T11:14:20.060295+0000","last_became_peered":"2024-09-16T11:14:20.060295+0000","last_unstale":"2024-09-16T11:14:50.643500+0000","last_undegraded":"2024-09-16T11:14:50.643500+0000","last_fullsized":"2024-09-16T11:14:50.643500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:30.845736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.643536+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:14:50.643536+0000","last_peered":"2024-09-16T11:14:50.643536+0000","last_clean":"2024-09-16T11:14:50.643536+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:14:50.643536+0000","last_undegraded":"2024-09-16T11:14:50.643536+0000","last_fullsized":"2024-09-16T11:14:50.643536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'448","reported_seq":2191,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.621604+0000","last_change":"2024-09-16T11:13:15.113157+0000","last_active":"2024-09-16T11:14:51.621604+0000","last_peered":"2024-09-16T11:14:51.621604+0000","last_clean":"2024-09-16T11:14:51.621604+0000","last_became_active":"2024-09-16T11:13:13.112648+0000","last_became_peered":"2024-09-16T11:13:13.112648+0000","last_unstale":"2024-09-16T11:14:51.621604+0000","last_undegraded":"2024-09-16T11:14:51.621604+0000","last_fullsized":"2024-09-16T11:14:51.621604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:52.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014276930000000001,"stat_sum":{"num_bytes":17686609,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":21,"num_read":3136,"num_read_kb":8294,"num_write":555,"num_write_kb":15978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"51'1","reported_seq":105,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.708674+0000","last_change":"2024-09-16T11:14:30.431871+0000","last_active":"2024-09-16T11:14:50.708674+0000","last_peered":"2024-09-16T11:14:50.708674+0000","last_clean":"2024-09-16T11:14:50.708674+0000","last_became_active":"2024-09-16T11:14:30.429187+0000","last_became_peered":"2024-09-16T11:14:30.429187+0000","last_unstale":"2024-09-16T11:14:50.708674+0000","last_undegraded":"2024-09-16T11:14:50.708674+0000","last_fullsized":"2024-09-16T11:14:50.708674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:42.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.611866+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:14:50.611866+0000","last_peered":"2024-09-16T11:14:50.611866+0000","last_clean":"2024-09-16T11:14:50.611866+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:14:50.611866+0000","last_undegraded":"2024-09-16T11:14:50.611866+0000","last_fullsized":"2024-09-16T11:14:50.611866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"107'239","reported_seq":2700,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.470540+0000","last_change":"2024-09-16T11:13:15.331158+0000","last_active":"2024-09-16T11:14:51.470540+0000","last_peered":"2024-09-16T11:14:51.470540+0000","last_clean":"2024-09-16T11:14:51.470540+0000","last_became_active":"2024-09-16T11:13:13.110085+0000","last_became_peered":"2024-09-16T11:13:13.110085+0000","last_unstale":"2024-09-16T11:14:51.470540+0000","last_undegraded":"2024-09-16T11:14:51.470540+0000","last_fullsized":"2024-09-16T11:14:51.470540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:43.994646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220874,"stat_sum":{"num_bytes":56613701,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":16,"num_read":2466,"num_read_kb":97507,"num_write":276,"num_write_kb":51265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"96'2","reported_seq":93,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.626303+0000","last_change":"2024-09-16T11:13:04.025567+0000","last_active":"2024-09-16T11:14:50.626303+0000","last_peered":"2024-09-16T11:14:50.626303+0000","last_clean":"2024-09-16T11:14:50.626303+0000","last_became_active":"2024-09-16T11:13:04.025252+0000","last_became_peered":"2024-09-16T11:13:04.025252+0000","last_unstale":"2024-09-16T11:14:50.626303+0000","last_undegraded":"2024-09-16T11:14:50.626303+0000","last_fullsized":"2024-09-16T11:14:50.626303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:43.890348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":129,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.622866+0000","last_change":"2024-09-16T11:14:20.067415+0000","last_active":"2024-09-16T11:14:50.622866+0000","last_peered":"2024-09-16T11:14:50.622866+0000","last_clean":"2024-09-16T11:14:50.622866+0000","last_became_active":"2024-09-16T11:14:20.064531+0000","last_became_peered":"2024-09-16T11:14:20.064531+0000","last_unstale":"2024-09-16T11:14:50.622866+0000","last_undegraded":"2024-09-16T11:14:50.622866+0000","last_fullsized":"2024-09-16T11:14:50.622866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:43.768064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":189,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.611789+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:14:50.611789+0000","last_peered":"2024-09-16T11:14:50.611789+0000","last_clean":"2024-09-16T11:14:50.611789+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:14:50.611789+0000","last_undegraded":"2024-09-16T11:14:50.611789+0000","last_fullsized":"2024-09-16T11:14:50.611789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"108'278","reported_seq":2109,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.656794+0000","last_change":"2024-09-16T11:14:34.480211+0000","last_active":"2024-09-16T11:14:51.656794+0000","last_peered":"2024-09-16T11:14:51.656794+0000","last_clean":"2024-09-16T11:14:51.656794+0000","last_became_active":"2024-09-16T11:14:34.443038+0000","last_became_peered":"2024-09-16T11:14:34.443038+0000","last_unstale":"2024-09-16T11:14:51.656794+0000","last_undegraded":"2024-09-16T11:14:51.656794+0000","last_fullsized":"2024-09-16T11:14:51.656794+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:00.846282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059007199999999997,"stat_sum":{"num_bytes":35441915,"num_objects":79,"num_object_clones":17,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":11,"num_read":1944,"num_read_kb":23110,"num_write":330,"num_write_kb":14213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":10240048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"107'426","reported_seq":1702,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.042910+0000","last_change":"2024-09-16T11:13:15.110036+0000","last_active":"2024-09-16T11:14:51.042910+0000","last_peered":"2024-09-16T11:14:51.042910+0000","last_clean":"2024-09-16T11:14:51.042910+0000","last_became_active":"2024-09-16T11:13:13.127166+0000","last_became_peered":"2024-09-16T11:13:13.127166+0000","last_unstale":"2024-09-16T11:14:51.042910+0000","last_undegraded":"2024-09-16T11:14:51.042910+0000","last_fullsized":"2024-09-16T11:14:51.042910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:20.304120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081336500000000003,"stat_sum":{"num_bytes":31583776,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":25,"num_read":2175,"num_read_kb":22005,"num_write":541,"num_write_kb":11223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"108'506","reported_seq":6327,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.684741+0000","last_change":"2024-09-16T11:14:33.450285+0000","last_active":"2024-09-16T11:14:51.684741+0000","last_peered":"2024-09-16T11:14:51.684741+0000","last_clean":"2024-09-16T11:14:51.684741+0000","last_became_active":"2024-09-16T11:14:33.449322+0000","last_became_peered":"2024-09-16T11:14:33.449322+0000","last_unstale":"2024-09-16T11:14:51.684741+0000","last_undegraded":"2024-09-16T11:14:51.684741+0000","last_fullsized":"2024-09-16T11:14:51.684741+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:32.119293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066151299999999997,"stat_sum":{"num_bytes":136276655,"num_objects":83,"num_object_clones":17,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":17,"num_read":7306,"num_read_kb":78498,"num_write":625,"num_write_kb":26317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":5354153,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"108'377","reported_seq":3595,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.630594+0000","last_change":"2024-09-16T11:13:15.134244+0000","last_active":"2024-09-16T11:14:51.630594+0000","last_peered":"2024-09-16T11:14:51.630594+0000","last_clean":"2024-09-16T11:14:51.630594+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:14:51.630594+0000","last_undegraded":"2024-09-16T11:14:51.630594+0000","last_fullsized":"2024-09-16T11:14:51.630594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:23.499921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054346600000000004,"stat_sum":{"num_bytes":48612497,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":17,"num_read":3490,"num_read_kb":44834,"num_write":443,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"107'342","reported_seq":5654,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.403679+0000","last_change":"2024-09-16T11:13:15.329809+0000","last_active":"2024-09-16T11:14:51.403679+0000","last_peered":"2024-09-16T11:14:51.403679+0000","last_clean":"2024-09-16T11:14:51.403679+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:14:51.403679+0000","last_undegraded":"2024-09-16T11:14:51.403679+0000","last_fullsized":"2024-09-16T11:14:51.403679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:36.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047034000000000003,"stat_sum":{"num_bytes":54863080,"num_objects":78,"num_object_clones":15,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":24,"num_read":5824,"num_read_kb":24778,"num_write":379,"num_write_kb":21175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"108'259","reported_seq":8753,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627639+0000","last_change":"2024-09-16T11:14:33.470081+0000","last_active":"2024-09-16T11:14:51.627639+0000","last_peered":"2024-09-16T11:14:51.627639+0000","last_clean":"2024-09-16T11:14:51.627639+0000","last_became_active":"2024-09-16T11:14:33.469151+0000","last_became_peered":"2024-09-16T11:14:33.469151+0000","last_unstale":"2024-09-16T11:14:51.627639+0000","last_undegraded":"2024-09-16T11:14:51.627639+0000","last_fullsized":"2024-09-16T11:14:51.627639+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023754140000000002,"stat_sum":{"num_bytes":31714617,"num_objects":70,"num_object_clones":13,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":20,"num_read":8396,"num_read_kb":36379,"num_write":286,"num_write_kb":17240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":15186808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"108'849","reported_seq":13896,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.634668+0000","last_change":"2024-09-16T11:14:33.439206+0000","last_active":"2024-09-16T11:14:51.634668+0000","last_peered":"2024-09-16T11:14:51.634668+0000","last_clean":"2024-09-16T11:14:51.634668+0000","last_became_active":"2024-09-16T11:14:33.437652+0000","last_became_peered":"2024-09-16T11:14:33.437652+0000","last_unstale":"2024-09-16T11:14:51.634668+0000","last_undegraded":"2024-09-16T11:14:51.634668+0000","last_fullsized":"2024-09-16T11:14:51.634668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:38.718306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025819319999999999,"stat_sum":{"num_bytes":64051884,"num_objects":84,"num_object_clones":18,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":23,"num_read":14342,"num_read_kb":57560,"num_write":1423,"num_write_kb":23216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19578786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"107'538","reported_seq":7899,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.534231+0000","last_change":"2024-09-16T11:14:34.444744+0000","last_active":"2024-09-16T11:14:51.534231+0000","last_peered":"2024-09-16T11:14:51.534231+0000","last_clean":"2024-09-16T11:14:51.534231+0000","last_became_active":"2024-09-16T11:14:34.443557+0000","last_became_peered":"2024-09-16T11:14:34.443557+0000","last_unstale":"2024-09-16T11:14:51.534231+0000","last_undegraded":"2024-09-16T11:14:51.534231+0000","last_fullsized":"2024-09-16T11:14:51.534231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:39.673819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00256719,"stat_sum":{"num_bytes":68530613,"num_objects":70,"num_object_clones":18,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":16,"num_read":9248,"num_read_kb":95120,"num_write":649,"num_write_kb":63765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":43055347,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"107'356","reported_seq":4194,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.527265+0000","last_change":"2024-09-16T11:14:44.551857+0000","last_active":"2024-09-16T11:14:51.527265+0000","last_peered":"2024-09-16T11:14:51.527265+0000","last_clean":"2024-09-16T11:14:51.527265+0000","last_became_active":"2024-09-16T11:14:44.547942+0000","last_became_peered":"2024-09-16T11:14:44.547942+0000","last_unstale":"2024-09-16T11:14:51.527265+0000","last_undegraded":"2024-09-16T11:14:51.527265+0000","last_fullsized":"2024-09-16T11:14:51.527265+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:05.099400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217035,"stat_sum":{"num_bytes":61587959,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":20,"num_read":4863,"num_read_kb":23609,"num_write":412,"num_write_kb":26997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":14969107,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1693094860,"num_objects":2403,"num_object_clones":495,"num_object_copies":4806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2403,"num_whiteouts":629,"num_read":194494,"num_read_kb":1301363,"num_write":20053,"num_write_kb":715899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1516,"num_bytes_recovered":218241032,"num_keys_recovered":271,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1108828160,"data_stored":1202046362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14895,"ondisk_log_size":14895,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416608,"kb_used_data":119244,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93955232,"statfs":{"total":96636764160,"available":96210157568,"internally_reserved":0,"allocated":122105856,"data_stored":128301044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1970000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0169999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.877,"15min":0.90100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2110000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.995,"15min":1.006},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.0449999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.371,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.502,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.024,"5min":1.0629999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.125,"15min":1.131},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1240000000000001,"15min":1.1339999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.82799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458812,"kb_used_data":148976,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93913028,"statfs":{"total":96636764160,"available":96166940672,"internally_reserved":0,"allocated":152551424,"data_stored":165266381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2430000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.296,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.089,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.872,"15min":0.874},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.81799999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.997,"15min":1.0149999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.198,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91200000000000003,"15min":0.90800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.353,"5min":1.466,"15min":1.466},"last":0.89900000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.014},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.371,"5min":1.417,"15min":1.417},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.22,"15min":1.1100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":15.597,"5min":15.597,"15min":15.597},"last":1.202}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.077,"15min":1.0900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5069999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.113}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.5,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.077,"15min":1.079},"min":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.3080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361340,"kb_used_data":51492,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94010500,"statfs":{"total":96636764160,"available":96266752000,"internally_reserved":0,"allocated":52727808,"data_stored":55999972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.48099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1819999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.501},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.873,"15min":0.88},"min":{"1min":0.53600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.143,"5min":1.397,"15min":1.397},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.48099999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89200000000000002,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1699999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.98299999999999998,"15min":1.012},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2290000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2909999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.747}]},{"osd":3,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90800000000000003,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.306,"5min":1.619,"15min":1.619},"last":0.79100000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.94899999999999995,"15min":0.94399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.034},{"interface":"front","average":{"1min":1.052,"5min":1.0069999999999999,"15min":0.999},"min":{"1min":0.55600000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.89900000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.96799999999999997,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.7450000000000001,"5min":1.2190000000000001,"15min":1.131},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.0880000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.081,"15min":1.077},"min":{"1min":0.77600000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.129,"15min":1.143},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7130000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1319999999999999}]}]},{"osd":4,"up_from":0,"seq":55834574877,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481184,"kb_used_data":175512,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93890656,"statfs":{"total":96636764160,"available":96144031744,"internally_reserved":0,"allocated":179724288,"data_stored":193441060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:13:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.30499999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1200000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":2.105},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.6699999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98799999999999999,"15min":0.997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":2.0569999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.371,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0369999999999999,"15min":1.03},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.131}]},{"osd":6,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.081,"15min":1.089},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":2.1440000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.77}]},{"osd":7,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0069999999999999,"15min":0.997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.349},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":2.1789999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420688,"kb_used_data":114956,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93951152,"statfs":{"total":96636764160,"available":96205979648,"internally_reserved":0,"allocated":117714944,"data_stored":133703763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.28,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1240000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.65100000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.64400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.496,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.238,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.115},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0660000000000001},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.0900000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.075},"min":{"1min":0.65200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.369,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.169},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.744,"15min":1.744},"last":1.0049999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.119,"15min":1.127},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.589,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.304,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":0.96599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532780,"kb_used_data":227108,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93839060,"statfs":{"total":96636764160,"available":96091197440,"internally_reserved":0,"allocated":232558592,"data_stored":252912385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.73399999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.94099999999999995}]},{"osd":4,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.075},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.306}]},{"osd":5,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.218},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.1970000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.1439999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.377},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.3400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437796,"kb_used_data":127952,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93934044,"statfs":{"total":96636764160,"available":96188461056,"internally_reserved":0,"allocated":131022848,"data_stored":138685076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.034,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4019999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":2.129,"15min":2.129},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0980000000000001,"15min":1.1220000000000001},"min":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3420000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.026},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.397,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.026},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4670000000000001,"5min":1.841,"15min":1.841},"last":0.38}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.034,"15min":1.042},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3839999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3600000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129142784,"data_stored":137466433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231268352,"data_stored":252284091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116428800,"data_stored":133075488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":178434048,"data_stored":192812702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":130629632,"data_stored":139315546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":51433472,"data_stored":55371595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":151265280,"data_stored":164638106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":120225792,"data_stored":127082401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:14:52.988 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:14:53.008 INFO:teuthology.orchestra.run.smithi162.stdout:4448 write 0x80d7851 thru 0x80e0668 (0x8e18 bytes) 2024-09-16T11:14:53.009 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:53.003+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:53.014 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.014 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.026 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:53.038 INFO:teuthology.orchestra.run.smithi162.stdout:4449 trunc from 0xe9263c3 to 0x65fa8c1 2024-09-16T11:14:53.069 INFO:teuthology.orchestra.run.smithi162.stdout:4450 read 0xf8f137 thru 0xf8f32c (0x1f6 bytes) 2024-09-16T11:14:53.071 INFO:teuthology.orchestra.run.smithi162.stdout:4451 read 0x1afdbbe thru 0x1b01bc0 (0x4003 bytes) 2024-09-16T11:14:53.089 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.089 INFO:teuthology.orchestra.run.smithi162.stdout:4455 writesame 0x1d5be6a thru 0x1d5dcac data_size 0x1(0x1e43 bytes) 2024-09-16T11:14:53.110 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:53.107+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:53.141 INFO:teuthology.orchestra.run.smithi162.stdout:4457 write 0xf5ca4f thru 0xf6390b (0x6ebd bytes) 2024-09-16T11:14:53.173 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.173 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.174 INFO:teuthology.orchestra.run.smithi162.stdout:4459 writesame 0x9b55e9d thru 0x9b64463 data_size 0x1(0xe5c7 bytes) 2024-09-16T11:14:53.174 INFO:teuthology.orchestra.run.smithi162.stdout:4461 write 0x3a22453 thru 0x3a2f908 (0xd4b6 bytes) 2024-09-16T11:14:53.184 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:53.190 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.190 INFO:teuthology.orchestra.run.smithi162.stdout:4466 write 0x182ef78 thru 0x183a1c2 (0xb24b bytes) 2024-09-16T11:14:53.214 INFO:teuthology.orchestra.run.smithi162.stdout:4469 read 0x8a4d0a3 thru 0x8a53313 (0x6271 bytes) 2024-09-16T11:14:53.214 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.214 INFO:teuthology.orchestra.run.smithi162.stdout:4471 read 0x8c26dcb thru 0x8c356b2 (0xe8e8 bytes) 2024-09-16T11:14:53.215 INFO:teuthology.orchestra.run.smithi162.stdout:4472 writesame 0x7f1c555 thru 0x7f1f7e9 data_size 0x1(0x3295 bytes) 2024-09-16T11:14:53.215 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:53.211+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:53.222 INFO:teuthology.orchestra.run.smithi162.stdout:4473 write 0x662455b thru 0x662c727 (0x81cd bytes) 2024-09-16T11:14:53.230 INFO:teuthology.orchestra.run.smithi162.stdout:4474 read 0x3de7a28 thru 0x3df2a70 (0xb049 bytes) 2024-09-16T11:14:53.268 INFO:teuthology.orchestra.run.smithi162.stdout:4475 write 0x9cacd75 thru 0x9cae82d (0x1ab9 bytes) 2024-09-16T11:14:53.279 INFO:teuthology.orchestra.run.smithi162.stdout:4476 writesame 0x812a14 thru 0x81c134 data_size 0x1(0x9721 bytes) 2024-09-16T11:14:53.311 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:53.305+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:53.319 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.319 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.329 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:53.330 INFO:teuthology.orchestra.run.smithi162.stdout:4480 trunc from 0x9cae82e to 0x8b8f3d7 2024-09-16T11:14:53.343 INFO:teuthology.orchestra.run.smithi162.stdout:4481 read 0x46ca2ce thru 0x46cf5c1 (0x52f4 bytes) 2024-09-16T11:14:53.349 INFO:teuthology.orchestra.run.smithi162.stdout:4482 write 0x8437a60 thru 0x843d938 (0x5ed9 bytes) 2024-09-16T11:14:53.374 INFO:teuthology.orchestra.run.smithi162.stdout:4484 trunc from 0x8b8f3d7 to 0xbd76f14 2024-09-16T11:14:53.381 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:14:53.382 INFO:teuthology.orchestra.run.smithi162.stdout:4485 write 0x50c0c11 thru 0x50c9212 (0x8602 bytes) 2024-09-16T11:14:53.385 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:14:53.401 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-16T11:14:52.626605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1693685330,"num_objects":2410,"num_object_clones":495,"num_object_copies":4820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":631,"num_read":194560,"num_read_kb":1301418,"num_write":20157,"num_write_kb":717530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1519,"num_bytes_recovered":218241032,"num_keys_recovered":271,"num_objects_omap":39,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14973,"ondisk_log_size":14973,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":3109208,"kb_used_data":965240,"kb_used_omap":11,"kb_used_meta":2143796,"kb_avail":657493672,"statfs":{"total":676457349120,"available":673273520128,"internally_reserved":0,"allocated":988405760,"data_stored":1068309681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2195247464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":185401918,"num_objects":222,"num_object_clones":40,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":90,"num_read":14725,"num_read_kb":137706,"num_write":961,"num_write_kb":64263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035720"},"pg_stats":[{"pgid":"4.1f","version":"108'284","reported_seq":5474,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.630784+0000","last_change":"2024-09-16T11:13:15.136003+0000","last_active":"2024-09-16T11:14:51.630784+0000","last_peered":"2024-09-16T11:14:51.630784+0000","last_clean":"2024-09-16T11:14:51.630784+0000","last_became_active":"2024-09-16T11:13:13.110861+0000","last_became_peered":"2024-09-16T11:13:13.110861+0000","last_unstale":"2024-09-16T11:14:51.630784+0000","last_undegraded":"2024-09-16T11:14:51.630784+0000","last_fullsized":"2024-09-16T11:14:51.630784+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:49.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002295123,"stat_sum":{"num_bytes":29854866,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":22,"num_read":6181,"num_read_kb":29712,"num_write":270,"num_write_kb":10157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"108'581","reported_seq":10209,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.652609+0000","last_change":"2024-09-16T11:13:15.329907+0000","last_active":"2024-09-16T11:14:51.652609+0000","last_peered":"2024-09-16T11:14:51.652609+0000","last_clean":"2024-09-16T11:14:51.652609+0000","last_became_active":"2024-09-16T11:13:13.129077+0000","last_became_peered":"2024-09-16T11:13:13.129077+0000","last_unstale":"2024-09-16T11:14:51.652609+0000","last_undegraded":"2024-09-16T11:14:51.652609+0000","last_fullsized":"2024-09-16T11:14:51.652609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:56.981990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050881799999999997,"stat_sum":{"num_bytes":64992645,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":17,"num_read":12166,"num_read_kb":90256,"num_write":781,"num_write_kb":41396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"108'351","reported_seq":4177,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.687197+0000","last_change":"2024-09-16T11:13:15.119814+0000","last_active":"2024-09-16T11:14:51.687197+0000","last_peered":"2024-09-16T11:14:51.687197+0000","last_clean":"2024-09-16T11:14:51.687197+0000","last_became_active":"2024-09-16T11:13:13.110835+0000","last_became_peered":"2024-09-16T11:13:13.110835+0000","last_unstale":"2024-09-16T11:14:51.687197+0000","last_undegraded":"2024-09-16T11:14:51.687197+0000","last_fullsized":"2024-09-16T11:14:51.687197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:25.687407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063997100000000003,"stat_sum":{"num_bytes":45525761,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":16,"num_read":4477,"num_read_kb":25852,"num_write":485,"num_write_kb":14109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"107'804","reported_seq":5107,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.534836+0000","last_change":"2024-09-16T11:14:24.176887+0000","last_active":"2024-09-16T11:14:51.534836+0000","last_peered":"2024-09-16T11:14:51.534836+0000","last_clean":"2024-09-16T11:14:51.534836+0000","last_became_active":"2024-09-16T11:14:24.105273+0000","last_became_peered":"2024-09-16T11:14:24.105273+0000","last_unstale":"2024-09-16T11:14:51.534836+0000","last_undegraded":"2024-09-16T11:14:51.534836+0000","last_fullsized":"2024-09-16T11:14:51.534836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:10.103253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038046360000000001,"stat_sum":{"num_bytes":38541162,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":23,"num_read":8028,"num_read_kb":29663,"num_write":1203,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2431957,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"107'349","reported_seq":4462,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.566038+0000","last_change":"2024-09-16T11:14:45.568220+0000","last_active":"2024-09-16T11:14:51.566038+0000","last_peered":"2024-09-16T11:14:51.566038+0000","last_clean":"2024-09-16T11:14:51.566038+0000","last_became_active":"2024-09-16T11:14:45.567373+0000","last_became_peered":"2024-09-16T11:14:45.567373+0000","last_unstale":"2024-09-16T11:14:51.566038+0000","last_undegraded":"2024-09-16T11:14:51.566038+0000","last_fullsized":"2024-09-16T11:14:51.566038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:19:25.613053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029419009999999998,"stat_sum":{"num_bytes":19443080,"num_objects":64,"num_object_clones":9,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":15,"num_read":4587,"num_read_kb":15973,"num_write":483,"num_write_kb":13985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":8560528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"108'428","reported_seq":3688,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.677326+0000","last_change":"2024-09-16T11:14:34.444229+0000","last_active":"2024-09-16T11:14:51.677326+0000","last_peered":"2024-09-16T11:14:51.677326+0000","last_clean":"2024-09-16T11:14:51.677326+0000","last_became_active":"2024-09-16T11:14:34.442783+0000","last_became_peered":"2024-09-16T11:14:34.442783+0000","last_unstale":"2024-09-16T11:14:51.677326+0000","last_undegraded":"2024-09-16T11:14:51.677326+0000","last_fullsized":"2024-09-16T11:14:51.677326+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:44:14.832997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024433810000000001,"stat_sum":{"num_bytes":44160458,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":16,"num_read":4352,"num_read_kb":31078,"num_write":559,"num_write_kb":20751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":17721559,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"107'390","reported_seq":2401,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.546450+0000","last_change":"2024-09-16T11:14:23.090655+0000","last_active":"2024-09-16T11:14:51.546450+0000","last_peered":"2024-09-16T11:14:51.546450+0000","last_clean":"2024-09-16T11:14:51.546450+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:14:51.546450+0000","last_undegraded":"2024-09-16T11:14:51.546450+0000","last_fullsized":"2024-09-16T11:14:51.546450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072097299999999997,"stat_sum":{"num_bytes":53879978,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":25,"num_read":3163,"num_read_kb":19399,"num_write":559,"num_write_kb":17721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"107'535","reported_seq":4689,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.124537+0000","last_change":"2024-09-16T11:14:34.443921+0000","last_active":"2024-09-16T11:14:51.124537+0000","last_peered":"2024-09-16T11:14:51.124537+0000","last_clean":"2024-09-16T11:14:51.124537+0000","last_became_active":"2024-09-16T11:14:34.443009+0000","last_became_peered":"2024-09-16T11:14:34.443009+0000","last_unstale":"2024-09-16T11:14:51.124537+0000","last_undegraded":"2024-09-16T11:14:51.124537+0000","last_fullsized":"2024-09-16T11:14:51.124537+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:23.629543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055416799999999996,"stat_sum":{"num_bytes":82359608,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":16,"num_read":6231,"num_read_kb":34918,"num_write":629,"num_write_kb":27883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":2375829,"num_keys_recovered":32,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"107'1411","reported_seq":6010,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.599688+0000","last_change":"2024-09-16T11:13:15.120697+0000","last_active":"2024-09-16T11:14:51.599688+0000","last_peered":"2024-09-16T11:14:51.599688+0000","last_clean":"2024-09-16T11:14:51.599688+0000","last_became_active":"2024-09-16T11:13:13.131852+0000","last_became_peered":"2024-09-16T11:13:13.131852+0000","last_unstale":"2024-09-16T11:14:51.599688+0000","last_undegraded":"2024-09-16T11:14:51.599688+0000","last_fullsized":"2024-09-16T11:14:51.599688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:23.813033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041000900000000002,"stat_sum":{"num_bytes":25509672,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":11625,"num_read_kb":16369,"num_write":2513,"num_write_kb":7973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"108'320","reported_seq":6519,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.636595+0000","last_change":"2024-09-16T11:13:15.330097+0000","last_active":"2024-09-16T11:14:51.636595+0000","last_peered":"2024-09-16T11:14:51.636595+0000","last_clean":"2024-09-16T11:14:51.636595+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:14:51.636595+0000","last_undegraded":"2024-09-16T11:14:51.636595+0000","last_fullsized":"2024-09-16T11:14:51.636595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061357700000000002,"stat_sum":{"num_bytes":59184687,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":21,"num_read":6964,"num_read_kb":49200,"num_write":396,"num_write_kb":15820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"107'312","reported_seq":2904,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.523249+0000","last_change":"2024-09-16T11:13:15.110275+0000","last_active":"2024-09-16T11:14:51.523249+0000","last_peered":"2024-09-16T11:14:51.523249+0000","last_clean":"2024-09-16T11:14:51.523249+0000","last_became_active":"2024-09-16T11:13:13.127081+0000","last_became_peered":"2024-09-16T11:13:13.127081+0000","last_unstale":"2024-09-16T11:14:51.523249+0000","last_undegraded":"2024-09-16T11:14:51.523249+0000","last_fullsized":"2024-09-16T11:14:51.523249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:29:58.185747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065955699999999996,"stat_sum":{"num_bytes":46667583,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":28,"num_read":3026,"num_read_kb":15217,"num_write":288,"num_write_kb":25173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"108'463","reported_seq":5355,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627741+0000","last_change":"2024-09-16T11:13:15.120212+0000","last_active":"2024-09-16T11:14:51.627741+0000","last_peered":"2024-09-16T11:14:51.627741+0000","last_clean":"2024-09-16T11:14:51.627741+0000","last_became_active":"2024-09-16T11:13:13.116260+0000","last_became_peered":"2024-09-16T11:13:13.116260+0000","last_unstale":"2024-09-16T11:14:51.627741+0000","last_undegraded":"2024-09-16T11:14:51.627741+0000","last_fullsized":"2024-09-16T11:14:51.627741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:15:32.165127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066315600000000003,"stat_sum":{"num_bytes":80787687,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":22,"num_read":6675,"num_read_kb":58352,"num_write":439,"num_write_kb":26774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"107'359","reported_seq":3224,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.808050+0000","last_change":"2024-09-16T11:14:44.550763+0000","last_active":"2024-09-16T11:14:50.808050+0000","last_peered":"2024-09-16T11:14:50.808050+0000","last_clean":"2024-09-16T11:14:50.808050+0000","last_became_active":"2024-09-16T11:14:44.549012+0000","last_became_peered":"2024-09-16T11:14:44.549012+0000","last_unstale":"2024-09-16T11:14:50.808050+0000","last_undegraded":"2024-09-16T11:14:50.808050+0000","last_fullsized":"2024-09-16T11:14:50.808050+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:38.040942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028235669999999999,"stat_sum":{"num_bytes":45646427,"num_objects":82,"num_object_clones":17,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":13,"num_read":3718,"num_read_kb":37178,"num_write":412,"num_write_kb":22646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":33382609,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"108'329","reported_seq":4389,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.621410+0000","last_change":"2024-09-16T11:14:08.511426+0000","last_active":"2024-09-16T11:14:51.621410+0000","last_peered":"2024-09-16T11:14:51.621410+0000","last_clean":"2024-09-16T11:14:51.621410+0000","last_became_active":"2024-09-16T11:14:08.510206+0000","last_became_peered":"2024-09-16T11:14:08.510206+0000","last_unstale":"2024-09-16T11:14:51.621410+0000","last_undegraded":"2024-09-16T11:14:51.621410+0000","last_fullsized":"2024-09-16T11:14:51.621410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:41.370533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145994,"stat_sum":{"num_bytes":83890873,"num_objects":75,"num_object_clones":15,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":16,"num_read":4449,"num_read_kb":51969,"num_write":581,"num_write_kb":22511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"107'490","reported_seq":3657,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627824+0000","last_change":"2024-09-16T11:13:15.119821+0000","last_active":"2024-09-16T11:14:51.627824+0000","last_peered":"2024-09-16T11:14:51.627824+0000","last_clean":"2024-09-16T11:14:51.627824+0000","last_became_active":"2024-09-16T11:13:13.131936+0000","last_became_peered":"2024-09-16T11:13:13.131936+0000","last_unstale":"2024-09-16T11:14:51.627824+0000","last_undegraded":"2024-09-16T11:14:51.627824+0000","last_fullsized":"2024-09-16T11:14:51.627824+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:24.053000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059135899999999996,"stat_sum":{"num_bytes":100058172,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":25,"num_read":4918,"num_read_kb":87602,"num_write":565,"num_write_kb":44087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"107'445","reported_seq":3918,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.545980+0000","last_change":"2024-09-16T11:14:45.570341+0000","last_active":"2024-09-16T11:14:51.545980+0000","last_peered":"2024-09-16T11:14:51.545980+0000","last_clean":"2024-09-16T11:14:51.545980+0000","last_became_active":"2024-09-16T11:14:45.569559+0000","last_became_peered":"2024-09-16T11:14:45.569559+0000","last_unstale":"2024-09-16T11:14:51.545980+0000","last_undegraded":"2024-09-16T11:14:51.545980+0000","last_fullsized":"2024-09-16T11:14:51.545980+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:51.205615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050522900000000005,"stat_sum":{"num_bytes":49539443,"num_objects":74,"num_object_clones":16,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":22,"num_read":4691,"num_read_kb":29800,"num_write":636,"num_write_kb":22416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":18357825,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.619688+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:14:50.619688+0000","last_peered":"2024-09-16T11:14:50.619688+0000","last_clean":"2024-09-16T11:14:50.619688+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:14:50.619688+0000","last_undegraded":"2024-09-16T11:14:50.619688+0000","last_fullsized":"2024-09-16T11:14:50.619688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"108'681","reported_seq":3716,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.658163+0000","last_change":"2024-09-16T11:14:33.441244+0000","last_active":"2024-09-16T11:14:51.658163+0000","last_peered":"2024-09-16T11:14:51.658163+0000","last_clean":"2024-09-16T11:14:51.658163+0000","last_became_active":"2024-09-16T11:14:33.439983+0000","last_became_peered":"2024-09-16T11:14:33.439983+0000","last_unstale":"2024-09-16T11:14:51.658163+0000","last_undegraded":"2024-09-16T11:14:51.658163+0000","last_fullsized":"2024-09-16T11:14:51.658163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:04.864478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071029199999999995,"stat_sum":{"num_bytes":35174535,"num_objects":81,"num_object_clones":16,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":27,"num_read":5882,"num_read_kb":16040,"num_write":882,"num_write_kb":7466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":3634035,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":102,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.614956+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:14:50.614956+0000","last_peered":"2024-09-16T11:14:50.614956+0000","last_clean":"2024-09-16T11:14:50.614956+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:14:50.614956+0000","last_undegraded":"2024-09-16T11:14:50.614956+0000","last_fullsized":"2024-09-16T11:14:50.614956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"108'302","reported_seq":4164,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627521+0000","last_change":"2024-09-16T11:13:15.119575+0000","last_active":"2024-09-16T11:14:51.627521+0000","last_peered":"2024-09-16T11:14:51.627521+0000","last_clean":"2024-09-16T11:14:51.627521+0000","last_became_active":"2024-09-16T11:13:13.131951+0000","last_became_peered":"2024-09-16T11:13:13.131951+0000","last_unstale":"2024-09-16T11:14:51.627521+0000","last_undegraded":"2024-09-16T11:14:51.627521+0000","last_fullsized":"2024-09-16T11:14:51.627521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:26.039221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051524000000000003,"stat_sum":{"num_bytes":47876326,"num_objects":83,"num_object_clones":18,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":27,"num_read":4182,"num_read_kb":35531,"num_write":324,"num_write_kb":11293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.611845+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:14:50.611845+0000","last_peered":"2024-09-16T11:14:50.611845+0000","last_clean":"2024-09-16T11:14:50.611845+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:14:50.611845+0000","last_undegraded":"2024-09-16T11:14:50.611845+0000","last_fullsized":"2024-09-16T11:14:50.611845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"108'763","reported_seq":10585,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.690315+0000","last_change":"2024-09-16T11:13:15.330268+0000","last_active":"2024-09-16T11:14:51.690315+0000","last_peered":"2024-09-16T11:14:51.690315+0000","last_clean":"2024-09-16T11:14:51.690315+0000","last_became_active":"2024-09-16T11:13:13.112956+0000","last_became_peered":"2024-09-16T11:13:13.112956+0000","last_unstale":"2024-09-16T11:14:51.690315+0000","last_undegraded":"2024-09-16T11:14:51.690315+0000","last_fullsized":"2024-09-16T11:14:51.690315+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:23:37.422681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674599999999998,"stat_sum":{"num_bytes":44856470,"num_objects":91,"num_object_clones":20,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":24,"num_read":12540,"num_read_kb":30045,"num_write":1082,"num_write_kb":20109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.708887+0000","last_change":"2024-09-16T11:13:52.391053+0000","last_active":"2024-09-16T11:14:50.708887+0000","last_peered":"2024-09-16T11:14:50.708887+0000","last_clean":"2024-09-16T11:14:50.708887+0000","last_became_active":"2024-09-16T11:13:52.390767+0000","last_became_peered":"2024-09-16T11:13:52.390767+0000","last_unstale":"2024-09-16T11:14:50.708887+0000","last_undegraded":"2024-09-16T11:14:50.708887+0000","last_fullsized":"2024-09-16T11:14:50.708887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:14.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"107'206","reported_seq":6537,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.446845+0000","last_change":"2024-09-16T11:13:15.112573+0000","last_active":"2024-09-16T11:14:51.446845+0000","last_peered":"2024-09-16T11:14:51.446845+0000","last_clean":"2024-09-16T11:14:51.446845+0000","last_became_active":"2024-09-16T11:13:13.113469+0000","last_became_peered":"2024-09-16T11:13:13.113469+0000","last_unstale":"2024-09-16T11:14:51.446845+0000","last_undegraded":"2024-09-16T11:14:51.446845+0000","last_fullsized":"2024-09-16T11:14:51.446845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:44.833246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815220000000001,"stat_sum":{"num_bytes":24016162,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":19,"num_read":6335,"num_read_kb":26900,"num_write":197,"num_write_kb":9360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.708965+0000","last_change":"2024-09-16T11:14:30.430965+0000","last_active":"2024-09-16T11:14:50.708965+0000","last_peered":"2024-09-16T11:14:50.708965+0000","last_clean":"2024-09-16T11:14:50.708965+0000","last_became_active":"2024-09-16T11:14:30.430629+0000","last_became_peered":"2024-09-16T11:14:30.430629+0000","last_unstale":"2024-09-16T11:14:50.708965+0000","last_undegraded":"2024-09-16T11:14:50.708965+0000","last_fullsized":"2024-09-16T11:14:50.708965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:33:44.285723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":104,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.622947+0000","last_change":"2024-09-16T11:12:56.834191+0000","last_active":"2024-09-16T11:14:50.622947+0000","last_peered":"2024-09-16T11:14:50.622947+0000","last_clean":"2024-09-16T11:14:50.622947+0000","last_became_active":"2024-09-16T11:12:54.819534+0000","last_became_peered":"2024-09-16T11:12:54.819534+0000","last_unstale":"2024-09-16T11:14:50.622947+0000","last_undegraded":"2024-09-16T11:14:50.622947+0000","last_fullsized":"2024-09-16T11:14:50.622947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:09:47.418906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"108'474","reported_seq":5915,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.653059+0000","last_change":"2024-09-16T11:14:23.105967+0000","last_active":"2024-09-16T11:14:51.653059+0000","last_peered":"2024-09-16T11:14:51.653059+0000","last_clean":"2024-09-16T11:14:51.653059+0000","last_became_active":"2024-09-16T11:14:23.092519+0000","last_became_peered":"2024-09-16T11:14:23.092519+0000","last_unstale":"2024-09-16T11:14:51.653059+0000","last_undegraded":"2024-09-16T11:14:51.653059+0000","last_fullsized":"2024-09-16T11:14:51.653059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:58.815065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001093002,"stat_sum":{"num_bytes":64165959,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":12,"num_read":7114,"num_read_kb":58615,"num_write":850,"num_write_kb":28192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":14007837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.643500+0000","last_change":"2024-09-16T11:14:20.060782+0000","last_active":"2024-09-16T11:14:50.643500+0000","last_peered":"2024-09-16T11:14:50.643500+0000","last_clean":"2024-09-16T11:14:50.643500+0000","last_became_active":"2024-09-16T11:14:20.060295+0000","last_became_peered":"2024-09-16T11:14:20.060295+0000","last_unstale":"2024-09-16T11:14:50.643500+0000","last_undegraded":"2024-09-16T11:14:50.643500+0000","last_fullsized":"2024-09-16T11:14:50.643500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:30.845736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.643536+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:14:50.643536+0000","last_peered":"2024-09-16T11:14:50.643536+0000","last_clean":"2024-09-16T11:14:50.643536+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:14:50.643536+0000","last_undegraded":"2024-09-16T11:14:50.643536+0000","last_fullsized":"2024-09-16T11:14:50.643536+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'448","reported_seq":2191,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.621604+0000","last_change":"2024-09-16T11:13:15.113157+0000","last_active":"2024-09-16T11:14:51.621604+0000","last_peered":"2024-09-16T11:14:51.621604+0000","last_clean":"2024-09-16T11:14:51.621604+0000","last_became_active":"2024-09-16T11:13:13.112648+0000","last_became_peered":"2024-09-16T11:13:13.112648+0000","last_unstale":"2024-09-16T11:14:51.621604+0000","last_undegraded":"2024-09-16T11:14:51.621604+0000","last_fullsized":"2024-09-16T11:14:51.621604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:51:52.289312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014276930000000001,"stat_sum":{"num_bytes":17686609,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":21,"num_read":3136,"num_read_kb":8294,"num_write":555,"num_write_kb":15978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"51'1","reported_seq":105,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.708674+0000","last_change":"2024-09-16T11:14:30.431871+0000","last_active":"2024-09-16T11:14:50.708674+0000","last_peered":"2024-09-16T11:14:50.708674+0000","last_clean":"2024-09-16T11:14:50.708674+0000","last_became_active":"2024-09-16T11:14:30.429187+0000","last_became_peered":"2024-09-16T11:14:30.429187+0000","last_unstale":"2024-09-16T11:14:50.708674+0000","last_undegraded":"2024-09-16T11:14:50.708674+0000","last_fullsized":"2024-09-16T11:14:50.708674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:47:42.961837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":101,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.611866+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:14:50.611866+0000","last_peered":"2024-09-16T11:14:50.611866+0000","last_clean":"2024-09-16T11:14:50.611866+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:14:50.611866+0000","last_undegraded":"2024-09-16T11:14:50.611866+0000","last_fullsized":"2024-09-16T11:14:50.611866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"107'239","reported_seq":2700,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.470540+0000","last_change":"2024-09-16T11:13:15.331158+0000","last_active":"2024-09-16T11:14:51.470540+0000","last_peered":"2024-09-16T11:14:51.470540+0000","last_clean":"2024-09-16T11:14:51.470540+0000","last_became_active":"2024-09-16T11:13:13.110085+0000","last_became_peered":"2024-09-16T11:13:13.110085+0000","last_unstale":"2024-09-16T11:14:51.470540+0000","last_undegraded":"2024-09-16T11:14:51.470540+0000","last_fullsized":"2024-09-16T11:14:51.470540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:43.994646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220874,"stat_sum":{"num_bytes":56613701,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":16,"num_read":2466,"num_read_kb":97507,"num_write":276,"num_write_kb":51265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"96'2","reported_seq":93,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.626303+0000","last_change":"2024-09-16T11:13:04.025567+0000","last_active":"2024-09-16T11:14:50.626303+0000","last_peered":"2024-09-16T11:14:50.626303+0000","last_clean":"2024-09-16T11:14:50.626303+0000","last_became_active":"2024-09-16T11:13:04.025252+0000","last_became_peered":"2024-09-16T11:13:04.025252+0000","last_unstale":"2024-09-16T11:14:50.626303+0000","last_undegraded":"2024-09-16T11:14:50.626303+0000","last_fullsized":"2024-09-16T11:14:50.626303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:43.890348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":129,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.622866+0000","last_change":"2024-09-16T11:14:20.067415+0000","last_active":"2024-09-16T11:14:50.622866+0000","last_peered":"2024-09-16T11:14:50.622866+0000","last_clean":"2024-09-16T11:14:50.622866+0000","last_became_active":"2024-09-16T11:14:20.064531+0000","last_became_peered":"2024-09-16T11:14:20.064531+0000","last_unstale":"2024-09-16T11:14:50.622866+0000","last_undegraded":"2024-09-16T11:14:50.622866+0000","last_fullsized":"2024-09-16T11:14:50.622866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:43.768064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":189,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:50.611789+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:14:50.611789+0000","last_peered":"2024-09-16T11:14:50.611789+0000","last_clean":"2024-09-16T11:14:50.611789+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:14:50.611789+0000","last_undegraded":"2024-09-16T11:14:50.611789+0000","last_fullsized":"2024-09-16T11:14:50.611789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"108'278","reported_seq":2109,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.656794+0000","last_change":"2024-09-16T11:14:34.480211+0000","last_active":"2024-09-16T11:14:51.656794+0000","last_peered":"2024-09-16T11:14:51.656794+0000","last_clean":"2024-09-16T11:14:51.656794+0000","last_became_active":"2024-09-16T11:14:34.443038+0000","last_became_peered":"2024-09-16T11:14:34.443038+0000","last_unstale":"2024-09-16T11:14:51.656794+0000","last_undegraded":"2024-09-16T11:14:51.656794+0000","last_fullsized":"2024-09-16T11:14:51.656794+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:00.846282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059007199999999997,"stat_sum":{"num_bytes":35441915,"num_objects":79,"num_object_clones":17,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":11,"num_read":1944,"num_read_kb":23110,"num_write":330,"num_write_kb":14213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":10240048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"107'426","reported_seq":1702,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.042910+0000","last_change":"2024-09-16T11:13:15.110036+0000","last_active":"2024-09-16T11:14:51.042910+0000","last_peered":"2024-09-16T11:14:51.042910+0000","last_clean":"2024-09-16T11:14:51.042910+0000","last_became_active":"2024-09-16T11:13:13.127166+0000","last_became_peered":"2024-09-16T11:13:13.127166+0000","last_unstale":"2024-09-16T11:14:51.042910+0000","last_undegraded":"2024-09-16T11:14:51.042910+0000","last_fullsized":"2024-09-16T11:14:51.042910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:23:20.304120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081336500000000003,"stat_sum":{"num_bytes":31583776,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":25,"num_read":2175,"num_read_kb":22005,"num_write":541,"num_write_kb":11223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"108'506","reported_seq":6327,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.684741+0000","last_change":"2024-09-16T11:14:33.450285+0000","last_active":"2024-09-16T11:14:51.684741+0000","last_peered":"2024-09-16T11:14:51.684741+0000","last_clean":"2024-09-16T11:14:51.684741+0000","last_became_active":"2024-09-16T11:14:33.449322+0000","last_became_peered":"2024-09-16T11:14:33.449322+0000","last_unstale":"2024-09-16T11:14:51.684741+0000","last_undegraded":"2024-09-16T11:14:51.684741+0000","last_fullsized":"2024-09-16T11:14:51.684741+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:32.119293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066151299999999997,"stat_sum":{"num_bytes":136276655,"num_objects":83,"num_object_clones":17,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":17,"num_read":7306,"num_read_kb":78498,"num_write":625,"num_write_kb":26317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":5354153,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"108'377","reported_seq":3595,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.630594+0000","last_change":"2024-09-16T11:13:15.134244+0000","last_active":"2024-09-16T11:14:51.630594+0000","last_peered":"2024-09-16T11:14:51.630594+0000","last_clean":"2024-09-16T11:14:51.630594+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:14:51.630594+0000","last_undegraded":"2024-09-16T11:14:51.630594+0000","last_fullsized":"2024-09-16T11:14:51.630594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:23.499921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054346600000000004,"stat_sum":{"num_bytes":48612497,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":17,"num_read":3490,"num_read_kb":44834,"num_write":443,"num_write_kb":18202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"107'342","reported_seq":5654,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.403679+0000","last_change":"2024-09-16T11:13:15.329809+0000","last_active":"2024-09-16T11:14:51.403679+0000","last_peered":"2024-09-16T11:14:51.403679+0000","last_clean":"2024-09-16T11:14:51.403679+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:14:51.403679+0000","last_undegraded":"2024-09-16T11:14:51.403679+0000","last_fullsized":"2024-09-16T11:14:51.403679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:36.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047034000000000003,"stat_sum":{"num_bytes":54863080,"num_objects":78,"num_object_clones":15,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":24,"num_read":5824,"num_read_kb":24778,"num_write":379,"num_write_kb":21175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"108'259","reported_seq":8753,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.627639+0000","last_change":"2024-09-16T11:14:33.470081+0000","last_active":"2024-09-16T11:14:51.627639+0000","last_peered":"2024-09-16T11:14:51.627639+0000","last_clean":"2024-09-16T11:14:51.627639+0000","last_became_active":"2024-09-16T11:14:33.469151+0000","last_became_peered":"2024-09-16T11:14:33.469151+0000","last_unstale":"2024-09-16T11:14:51.627639+0000","last_undegraded":"2024-09-16T11:14:51.627639+0000","last_fullsized":"2024-09-16T11:14:51.627639+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.712421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023754140000000002,"stat_sum":{"num_bytes":31714617,"num_objects":70,"num_object_clones":13,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":20,"num_read":8396,"num_read_kb":36379,"num_write":286,"num_write_kb":17240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":15186808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"108'849","reported_seq":13896,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.634668+0000","last_change":"2024-09-16T11:14:33.439206+0000","last_active":"2024-09-16T11:14:51.634668+0000","last_peered":"2024-09-16T11:14:51.634668+0000","last_clean":"2024-09-16T11:14:51.634668+0000","last_became_active":"2024-09-16T11:14:33.437652+0000","last_became_peered":"2024-09-16T11:14:33.437652+0000","last_unstale":"2024-09-16T11:14:51.634668+0000","last_undegraded":"2024-09-16T11:14:51.634668+0000","last_fullsized":"2024-09-16T11:14:51.634668+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:43:38.718306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025819319999999999,"stat_sum":{"num_bytes":64051884,"num_objects":84,"num_object_clones":18,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":23,"num_read":14342,"num_read_kb":57560,"num_write":1423,"num_write_kb":23216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":19578786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"107'538","reported_seq":7899,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.534231+0000","last_change":"2024-09-16T11:14:34.444744+0000","last_active":"2024-09-16T11:14:51.534231+0000","last_peered":"2024-09-16T11:14:51.534231+0000","last_clean":"2024-09-16T11:14:51.534231+0000","last_became_active":"2024-09-16T11:14:34.443557+0000","last_became_peered":"2024-09-16T11:14:34.443557+0000","last_unstale":"2024-09-16T11:14:51.534231+0000","last_undegraded":"2024-09-16T11:14:51.534231+0000","last_fullsized":"2024-09-16T11:14:51.534231+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:39.673819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00256719,"stat_sum":{"num_bytes":68530613,"num_objects":70,"num_object_clones":18,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":16,"num_read":9248,"num_read_kb":95120,"num_write":649,"num_write_kb":63765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":43055347,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"107'356","reported_seq":4194,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-16T11:14:51.527265+0000","last_change":"2024-09-16T11:14:44.551857+0000","last_active":"2024-09-16T11:14:51.527265+0000","last_peered":"2024-09-16T11:14:51.527265+0000","last_clean":"2024-09-16T11:14:51.527265+0000","last_became_active":"2024-09-16T11:14:44.547942+0000","last_became_peered":"2024-09-16T11:14:44.547942+0000","last_unstale":"2024-09-16T11:14:51.527265+0000","last_undegraded":"2024-09-16T11:14:51.527265+0000","last_fullsized":"2024-09-16T11:14:51.527265+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:05.099400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217035,"stat_sum":{"num_bytes":61587959,"num_objects":69,"num_object_clones":12,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":20,"num_read":4863,"num_read_kb":23609,"num_write":412,"num_write_kb":26997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":14969107,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1693094860,"num_objects":2403,"num_object_clones":495,"num_object_copies":4806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2403,"num_whiteouts":629,"num_read":194494,"num_read_kb":1301363,"num_write":20053,"num_write_kb":715899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1516,"num_bytes_recovered":218241032,"num_keys_recovered":271,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1108828160,"data_stored":1202046362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14895,"ondisk_log_size":14895,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416608,"kb_used_data":119244,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93955232,"statfs":{"total":96636764160,"available":96210157568,"internally_reserved":0,"allocated":122105856,"data_stored":128301044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.503}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1970000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0169999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.877,"15min":0.90100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.59099999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2110000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.995,"15min":1.006},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.0449999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.371,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.502,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.024,"5min":1.0629999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.1259999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.125,"15min":1.131},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1240000000000001,"15min":1.1339999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.82799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458812,"kb_used_data":148976,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93913028,"statfs":{"total":96636764160,"available":96166940672,"internally_reserved":0,"allocated":152551424,"data_stored":165266381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2430000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.296,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.089,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.872,"15min":0.874},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.81799999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.997,"15min":1.0149999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.198,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91200000000000003,"15min":0.90800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.353,"5min":1.466,"15min":1.466},"last":0.89900000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.014},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.371,"5min":1.417,"15min":1.417},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.22,"15min":1.1100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":15.597,"5min":15.597,"15min":15.597},"last":1.202}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.077,"15min":1.0900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5069999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.113}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.5,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.077,"15min":1.079},"min":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.3080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361340,"kb_used_data":51492,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94010500,"statfs":{"total":96636764160,"available":96266752000,"internally_reserved":0,"allocated":52727808,"data_stored":55999972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.84099999999999997,"15min":0.85499999999999998},"min":{"1min":0.48099999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1819999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.501},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.873,"15min":0.88},"min":{"1min":0.53600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.143,"5min":1.397,"15min":1.397},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.48099999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89200000000000002,"15min":0.90100000000000002},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1699999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.98299999999999998,"15min":1.012},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2290000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2909999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.747}]},{"osd":3,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90800000000000003,"15min":0.90700000000000003},"min":{"1min":0.65300000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.92200000000000004,"15min":0.92800000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.306,"5min":1.619,"15min":1.619},"last":0.79100000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.94899999999999995,"15min":0.94399999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.034},{"interface":"front","average":{"1min":1.052,"5min":1.0069999999999999,"15min":0.999},"min":{"1min":0.55600000000000005,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.89900000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.96799999999999997,"15min":0.97099999999999997},"min":{"1min":0.58099999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.7450000000000001,"5min":1.2190000000000001,"15min":1.131},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":16.024000000000001,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.0880000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:52 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.081,"15min":1.077},"min":{"1min":0.77600000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.129,"15min":1.143},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.7130000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1319999999999999}]}]},{"osd":4,"up_from":0,"seq":55834574877,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":481184,"kb_used_data":175512,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93890656,"statfs":{"total":96636764160,"available":96144031744,"internally_reserved":0,"allocated":179724288,"data_stored":193441060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:13:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.30499999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1200000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":2.105},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.6699999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98799999999999999,"15min":0.997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":2.0569999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.371,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0369999999999999,"15min":1.03},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.131}]},{"osd":6,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.081,"15min":1.089},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":2.1440000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.77}]},{"osd":7,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0069999999999999,"15min":0.997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.349},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":2.1789999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420688,"kb_used_data":114956,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93951152,"statfs":{"total":96636764160,"available":96205979648,"internally_reserved":0,"allocated":117714944,"data_stored":133703763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.28,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1240000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.65100000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.64400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.496,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.238,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.0589999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.115},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0660000000000001},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.0900000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.075},"min":{"1min":0.65200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.369,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.169},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.744,"15min":1.744},"last":1.0049999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.119,"15min":1.127},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.589,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.304,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":0.96599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532780,"kb_used_data":227108,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93839060,"statfs":{"total":96636764160,"available":96091197440,"internally_reserved":0,"allocated":232558592,"data_stored":252912385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.73399999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.94099999999999995}]},{"osd":4,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.075},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.306}]},{"osd":5,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.218},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.266,"5min":2.266,"15min":2.266},"last":1.1970000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.1439999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:13:49 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.377},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.6600000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.3400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437796,"kb_used_data":127952,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93934044,"statfs":{"total":96636764160,"available":96188461056,"internally_reserved":0,"allocated":131022848,"data_stored":138685076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.034,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.33800000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4019999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":2.129,"15min":2.129},"last":0.67900000000000005}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0980000000000001,"15min":1.1220000000000001},"min":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3420000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.026},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.397,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.026},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4670000000000001,"5min":1.841,"15min":1.841},"last":0.38}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.034,"15min":1.042},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3839999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3600000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129142784,"data_stored":137466433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231268352,"data_stored":252284091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116428800,"data_stored":133075488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":178434048,"data_stored":192812702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":130629632,"data_stored":139315546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":51433472,"data_stored":55371595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":151265280,"data_stored":164638106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":120225792,"data_stored":127082401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:14:53.404 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T11:14:53.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:53.404 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 3, 5, 6, 7, 2] 2024-09-16T11:14:53.404 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-16T11:14:53.405 INFO:teuthology.orchestra.run.smithi162.stdout:4486 writesame 0x5b7db38 thru 0x5b895d4 data_size 0x2b(0xba9d bytes) 2024-09-16T11:14:53.413 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:53.405+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:53.445 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.445 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.447 INFO:teuthology.orchestra.run.smithi162.stdout:4490 punch from 0x50c620d to 0x50d4fef, (0xede2 bytes) 2024-09-16T11:14:53.447 INFO:teuthology.orchestra.run.smithi162.stdout:4491 writesame 0x6b93e92 thru 0x6b99b72 data_size 0x1f(0x5ce1 bytes) 2024-09-16T11:14:53.454 INFO:teuthology.orchestra.run.smithi162.stdout:4492 punch from 0x5332288 to 0x533f0e5, (0xce5d bytes) 2024-09-16T11:14:53.454 INFO:teuthology.orchestra.run.smithi162.stdout:4493 writesame 0x28587bc thru 0x285e5f8 data_size 0x5(0x5e3d bytes) 2024-09-16T11:14:53.455 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:53.472 INFO:teuthology.orchestra.run.smithi162.stdout:4494 trunc from 0xbd76f14 to 0xb5fa173 2024-09-16T11:14:53.479 INFO:teuthology.orchestra.run.smithi162.stdout:4495 writesame 0x4a960cf thru 0x4a9b413 data_size 0x1(0x5345 bytes) 2024-09-16T11:14:53.503 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.503 INFO:teuthology.orchestra.run.smithi162.stdout:4497 writesame 0x8b11d5d thru 0x8b1d32b data_size 0x3d(0xb5cf bytes) 2024-09-16T11:14:53.510 INFO:teuthology.orchestra.run.smithi162.stdout:4498 punch from 0x1cffd8d to 0x1d09190, (0x9403 bytes) 2024-09-16T11:14:53.510 INFO:teuthology.orchestra.run.smithi162.stdout:4500 writesame 0x375d76c thru 0x3762baa data_size 0x4f(0x543f bytes) 2024-09-16T11:14:53.513 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:53.505+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:53.522 INFO:teuthology.orchestra.run.smithi162.stdout:4505 read 0x8c05483 thru 0x8c128c9 (0xd447 bytes) 2024-09-16T11:14:53.522 INFO:teuthology.orchestra.run.smithi162.stdout:4506 write 0xa326095 thru 0xa333f09 (0xde75 bytes) 2024-09-16T11:14:53.546 INFO:teuthology.orchestra.run.smithi162.stdout:4507 trunc from 0xb5fa173 to 0x122b2ee 2024-09-16T11:14:53.550 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.550 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.560 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:53.608 INFO:teuthology.orchestra.run.smithi162.stdout:4508 read 0x111d990 thru 0x111ef61 (0x15d2 bytes) 2024-09-16T11:14:53.614 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:53.609+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:53.649 INFO:teuthology.orchestra.run.smithi162.stdout:4509 writesame 0x6604f88 thru 0x6611e73 data_size 0x52(0xceec bytes) 2024-09-16T11:14:53.649 INFO:teuthology.orchestra.run.smithi162.stdout:4510 read 0x7b6e97 thru 0x7ba213 (0x337d bytes) 2024-09-16T11:14:53.649 INFO:teuthology.orchestra.run.smithi162.stdout:4512 writesame 0x1b35aa4 thru 0x1b3a5c5 data_size 0x3b(0x4b22 bytes) 2024-09-16T11:14:53.654 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.654 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.661 INFO:teuthology.orchestra.run.smithi162.stdout:4513 punch from 0x2cd20e1 to 0x2cd2e50, (0xd6f bytes) 2024-09-16T11:14:53.661 INFO:teuthology.orchestra.run.smithi162.stdout:4514 punch from 0x5f27a20 to 0x5f28064, (0x644 bytes) 2024-09-16T11:14:53.661 INFO:teuthology.orchestra.run.smithi162.stdout:4515 writesame 0xfee050 thru 0xff01a5 data_size 0x11(0x2156 bytes) 2024-09-16T11:14:53.663 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:53.694 INFO:teuthology.orchestra.run.smithi162.stdout:4516 write 0x620cdf4 thru 0x6217be1 (0xadee bytes) 2024-09-16T11:14:53.702 INFO:teuthology.orchestra.run.smithi162.stdout:4517 trunc from 0x6611e74 to 0x62e6017 2024-09-16T11:14:53.713 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.713 INFO:teuthology.orchestra.run.smithi162.stdout:4519 punch from 0x164aae6 to 0x1659be6, (0xf100 bytes) 2024-09-16T11:14:53.714 INFO:teuthology.orchestra.run.smithi162.stdout:4520 trunc from 0x62e6017 to 0x7b4ec3f 2024-09-16T11:14:53.715 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:53.709+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:53.721 INFO:teuthology.orchestra.run.smithi162.stdout:4521 trunc from 0x7b4ec3f to 0x73bf646 2024-09-16T11:14:53.729 INFO:teuthology.orchestra.run.smithi162.stdout:4522 read 0x489748 thru 0x490d8b (0x7644 bytes) 2024-09-16T11:14:53.740 INFO:teuthology.orchestra.run.smithi162.stdout:4525 writesame 0x41a8009 thru 0x41aa8fe data_size 0xe(0x28f6 bytes) 2024-09-16T11:14:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:53.780 INFO:teuthology.orchestra.run.smithi162.stdout:4526 write 0xba60695 thru 0xba64adf (0x444b bytes) 2024-09-16T11:14:53.802 INFO:teuthology.orchestra.run.smithi162.stdout:4527 writesame 0xee4caf7 thru 0xee56aa0 data_size 0x2(0x9faa bytes) 2024-09-16T11:14:53.802 INFO:teuthology.orchestra.run.smithi162.stdout:4528 writesame 0x7ae3b27 thru 0x7ae3f16 data_size 0x1c(0x3f0 bytes) 2024-09-16T11:14:53.806 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:14:53.803+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e109 definitely_dead 0 2024-09-16T11:14:53.808 INFO:teuthology.orchestra.run.smithi162.stdout:4529 read 0xbf7b89f thru 0xbf8ad94 (0xf4f6 bytes) 2024-09-16T11:14:53.808 INFO:teuthology.orchestra.run.smithi162.stdout:4530 write 0x9a7edb2 thru 0x9a889b4 (0x9c03 bytes) 2024-09-16T11:14:53.818 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:53.813+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:53.822 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.822 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.822 INFO:teuthology.orchestra.run.smithi162.stdout:4535 write 0x1a81c2f thru 0x1a8a592 (0x8964 bytes) 2024-09-16T11:14:53.829 INFO:teuthology.orchestra.run.smithi162.stdout:4536 writesame 0xb7bb62e thru 0xb7c780c data_size 0x1(0xc1df bytes) 2024-09-16T11:14:53.853 INFO:teuthology.orchestra.run.smithi162.stdout:4538 write 0x599c063 thru 0x59a580f (0x97ad bytes) 2024-09-16T11:14:53.861 INFO:teuthology.orchestra.run.smithi162.stdout:4539 read 0x2aa3bff thru 0x2aaaf4d (0x734f bytes) 2024-09-16T11:14:53.861 INFO:teuthology.orchestra.run.smithi162.stdout:4540 read 0x13e0f58 thru 0x13ea0d2 (0x917b bytes) 2024-09-16T11:14:53.861 INFO:teuthology.orchestra.run.smithi162.stdout:4542 read 0xc114869 thru 0xc1151e3 (0x97b bytes) 2024-09-16T11:14:53.861 INFO:teuthology.orchestra.run.smithi162.stdout:4543 write 0x653c2fd thru 0x654b270 (0xef74 bytes) 2024-09-16T11:14:53.867 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.867 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.874 INFO:teuthology.orchestra.run.smithi162.stdout:4544 read 0x6b335d2 thru 0x6b3414f (0xb7e bytes) 2024-09-16T11:14:53.876 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:53.889 INFO:teuthology.orchestra.run.smithi162.stdout:4547 trunc from 0xee56aa1 to 0x81649a7 2024-09-16T11:14:53.904 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:53.904 INFO:teuthology.orchestra.run.smithi162.stdout:4550 read 0x7c2d008 thru 0x7c31ff5 (0x4fee bytes) 2024-09-16T11:14:53.916 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:53.909+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:53.921 INFO:teuthology.orchestra.run.smithi162.stdout:4551 write 0xcffedff thru 0xd000b22 (0x1d24 bytes) 2024-09-16T11:14:53.931 INFO:teuthology.orchestra.run.smithi162.stdout:4552 write 0x29460b5 thru 0x294ef42 (0x8e8e bytes) 2024-09-16T11:14:53.957 INFO:teuthology.orchestra.run.smithi162.stdout:4553 trunc from 0xd000b23 to 0x480e7ed 2024-09-16T11:14:53.972 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:53.972 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:53.975 INFO:teuthology.orchestra.run.smithi162.stdout:4554 writesame 0x3d43306 thru 0x3d460de data_size 0xb(0x2dd9 bytes) 2024-09-16T11:14:53.981 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:53.986 INFO:teuthology.orchestra.run.smithi162.stdout:4555 clone 38 order 18 su 32768 sc 6 2024-09-16T11:14:54.018 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:54.015+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:54.121 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:54.117+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:54.137 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:54.138 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:54.147 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:54.222 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:54.219+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:54.248 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:54.248 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:54.257 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:54.323 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:54.317+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:54.358 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:54.358 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:54.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:54.367 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:54.423 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:54.417+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:54.468 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:54.469 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:54.477 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:54.512 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:54.524 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:54.519+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:54.532 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:54.554 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:54.571 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:54.572 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:54.572 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:54.581 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:54.591 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:54.610 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:54.625 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:54.617+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:54.632 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:54.657 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:14:54.665 INFO:teuthology.orchestra.run.smithi162.stderr:marked out osd.5. 2024-09-16T11:14:54.666 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:54.669 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:54.687 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:54.687 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:54.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.908272564, 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:14:54.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:14:54.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:14:54.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:14:54.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:54.695 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.520346864, 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:54.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:14:54.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.520195534, 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.701 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:14:54.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.707 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.52014404800001, 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.708 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.5200739, 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.714 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:14:54.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:14:54.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.51990143, 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:14:54.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:14:54.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:14:54.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.519843108, 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.727 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:14:54.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:14:54.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.51969098799999, 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:14:54.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:14:54.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.51951246900001, 2024-09-16T11:14:54.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:14:54.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:14:54.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:54.745 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.519479496, 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:14:54.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:14:54.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.751 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.96729177100001, 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.752 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:14:54.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:14:54.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.967162974, 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:14:54.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:14:54.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.967104629, 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.764 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:14:54.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.770 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.96705024000001, 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.771 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:14:54.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.966915933, 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:14:54.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:54.782 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.18576056400001, 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:14:54.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.185673252, 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.788 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:14:54.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.18543512399999, 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.795 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:14:54.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:14:54.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.800 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.505055114999998, 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:54.805 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.796862679, 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:14:54.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:54.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:14:54.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:14:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:54.810 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:54.810 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:54.811 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:54.805+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:54.812 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:54.910 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:54.911 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:54.911 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:54.911 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:54.912 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:54.905+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:54.919 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:55.013 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:55.011+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:55.016 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.017 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:55.017 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:55.017 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:55.017 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.026 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:55.114 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:55.111+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:55.172 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.993861960999993, 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:14:55.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:14:55.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42162 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 245760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.993784218000002, 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062313320999999998, 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42162 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174355+0000", 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174356+0000", 2024-09-16T11:14:55.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0200000000000002e-07 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174358+0000", 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2709999999999999e-06 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174364+0000", 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7010000000000001e-06 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174367+0000", 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.535e-06 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236531+0000", 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062163389999999999 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236594+0000", 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3880000000000005e-05 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236668+0000", 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4041999999999993e-05 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42182 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.991418202999995, 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062582578, 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42182 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176721+0000", 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176722+0000", 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5300000000000004e-07 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176725+0000", 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8150000000000002e-06 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176731+0000", 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4289999999999998e-06 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176736+0000", 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.383e-06 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238421+0000", 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061684655999999997 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.238609+0000", 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018781099999999999 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239304+0000", 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00069483100000000003 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.991398687, 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:14:55.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42184 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.990261183000001, 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062716243000000005, 2024-09-16T11:14:55.191 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42184 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177878+0000", 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177879+0000", 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6499999999999999e-07 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177881+0000", 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1679999999999998e-06 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177887+0000", 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9120000000000003e-06 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177890+0000", 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4920000000000002e-06 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240242+0000", 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062351756000000001 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240332+0000", 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9820000000000004e-05 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240594+0000", 2024-09-16T11:14:55.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026263000000000002 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42185 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.990245283999997, 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062534720000000002, 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42185 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177894+0000", 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177895+0000", 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5600000000000001e-07 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177901+0000", 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6599999999999998e-06 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177907+0000", 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.032e-06 2024-09-16T11:14:55.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177914+0000", 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.849e-06 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240267+0000", 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062352672999999997 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240348+0000", 2024-09-16T11:14:55.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1112999999999994e-05 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240429+0000", 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0937000000000001e-05 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42186 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.990221030000001, 2024-09-16T11:14:55.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063405550000000005, 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42186 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177918+0000", 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177919+0000", 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.58e-07 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177921+0000", 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2029999999999999e-06 2024-09-16T11:14:55.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177927+0000", 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6099999999999997e-06 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177930+0000", 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.225e-06 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240617+0000", 2024-09-16T11:14:55.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062687539 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240970+0000", 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035288700000000002 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241324+0000", 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035362799999999998 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42187 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.990205771000007, 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062927635999999995, 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42187 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.205 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177934+0000", 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177936+0000", 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4509999999999999e-06 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177942+0000", 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.468e-06 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177945+0000", 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3529999999999999e-06 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240690+0000", 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062745144000000003 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240773+0000", 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2589000000000005e-05 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240861+0000", 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8264999999999996e-05 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.990190451999993, 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:14:55.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42257 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988168048000006, 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411951, 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42257 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179971+0000", 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179972+0000", 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9200000000000002e-07 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179974+0000", 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.863e-06 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179979+0000", 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5809999999999996e-06 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179982+0000", 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.039e-06 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242082+0000", 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062099513000000002 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242148+0000", 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6707999999999999e-05 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242383+0000", 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023485499999999999 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42258 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988153550000007, 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062831919, 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42258 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179986+0000", 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:55.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179988+0000", 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8840000000000001e-06 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179993+0000", 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3759999999999999e-06 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.179997+0000", 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2459999999999999e-06 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242408+0000", 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062411038000000002 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242475+0000", 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7526000000000004e-05 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242818+0000", 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034247599999999998 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42259 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988139216999997, 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063330388000000001, 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.223 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42259 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180000+0000", 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180001+0000", 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180002+0000", 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7999999999999999e-06 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180008+0000", 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4659999999999998e-06 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180011+0000", 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0199999999999999e-06 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242847+0000", 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062836639 2024-09-16T11:14:55.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.242929+0000", 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1434000000000007e-05 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243331+0000", 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040165600000000002 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988125159999996, 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988111304, 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:14:55.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988097429000007, 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:14:55.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:14:55.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.241 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988083661999994, 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:14:55.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:14:55.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:55.246 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.988036152999996, 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:55.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:14:55.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.987750335000001, 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:14:55.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.255 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.895086519000003, 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:14:55.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:14:55.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.894937337000002, 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:14:55.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:55.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.267 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.269 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:55.269 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:55.261+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:55.370 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:55.365+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4.acdb2a47 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.71128731599999995, 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.71133175500000001, 2024-09-16T11:14:55.375 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:14:55.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:14:55.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:14:55.379 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:55.438 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:14:55.470 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:55.467+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:55.487 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.487 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:55.488 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:55.488 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:55.572 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:55.565+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.100880064000002, 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.598 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:14:55.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:14:55.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99422 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 776192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.851095430999997, 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11283726099999999, 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.606 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99422 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742458+0000", 2024-09-16T11:14:55.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742911+0000", 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045312299999999999 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742922+0000", 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1068e-05 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742939+0000", 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6685e-05 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.742955+0000", 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6419e-05 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855055+0000", 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11209968100000001 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855150+0000", 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5185000000000004e-05 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.855295+0000", 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001451 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99524 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 587776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.850000535, 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112829479, 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99524 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.743553+0000", 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744021+0000", 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046845900000000001 2024-09-16T11:14:55.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744026+0000", 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5889999999999996e-06 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744039+0000", 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3326000000000001e-05 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.744045+0000", 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9460000000000003e-06 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856169+0000", 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112124052 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856233+0000", 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4344000000000006e-05 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.856382+0000", 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000148763 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99537 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.847487299999997, 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11344878899999999, 2024-09-16T11:14:55.615 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99537 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746066+0000", 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4899999999999995e-07 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746069+0000", 2024-09-16T11:14:55.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5569999999999998e-06 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746077+0000", 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1440000000000003e-06 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746081+0000", 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3540000000000002e-06 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.857981+0000", 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.111899208 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.858055+0000", 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4374000000000003e-05 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859515+0000", 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0014596030000000001 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.619 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.847467266000002, 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.620 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:14:55.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:55.624 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.847197950000002, 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:55.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:14:55.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.846799439999998, 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:14:55.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.846510651999999, 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:14:55.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99546 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.844899376999997, 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11333913599999999, 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99546 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.638 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748654+0000", 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748655+0000", 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5899999999999995e-07 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748665+0000", 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0776e-05 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748714+0000", 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8337999999999997e-05 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748720+0000", 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1709999999999999e-06 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861601+0000", 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112880861 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861743+0000", 2024-09-16T11:14:55.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014191 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861993+0000", 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025022100000000002 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:55.642 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.844811235999998, 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:14:55.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.844732919999998, 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:14:55.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:14:55.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.844699181000003, 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:14:55.651 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:14:55.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.844658436000003, 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:14:55.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99563 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 630784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.841467993999998, 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112716318, 2024-09-16T11:14:55.660 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99563 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752085+0000", 2024-09-16T11:14:55.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7799999999999997e-07 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752087+0000", 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.314e-06 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752091+0000", 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752093+0000", 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2170000000000001e-06 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864490+0000", 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11239703700000001 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864596+0000", 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0001063 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864801+0000", 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000205038 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.841457988000002, 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:14:55.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:14:55.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.841439930999996, 2024-09-16T11:14:55.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:14:55.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.841413942000003, 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:14:55.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:14:55.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.840715510999999, 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:14:55.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.840702952000001, 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:14:55.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99591 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.837013530999997, 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113533827, 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99591 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:55.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756540+0000", 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756541+0000", 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5200000000000001e-06 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756545+0000", 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9450000000000003e-06 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756555+0000", 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5070000000000006e-06 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869736+0000", 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11318076000000001 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.869888+0000", 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015207900000000001 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.870073+0000", 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018570899999999999 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:55.692 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:55.694 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:55.691+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:55.795 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:55.789+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:55.803 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.803 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:55.803 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:55.803 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.813 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:55.896 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:55.891+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:55.916 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:55.916 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:55.916 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:55.916 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:55.916 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:55.926 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:55.997 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:55.993+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955642062999999, 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:14:56.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:14:56.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955627604999997, 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:14:56.043 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:14:56.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:14:56.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955610405000002, 2024-09-16T11:14:56.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:56.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:14:56.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:14:56.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955596884999999, 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:14:56.054 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.059 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955588343000002, 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:14:56.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:14:56.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:14:56.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:14:56.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955578199000001, 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:14:56.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:14:56.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:14:56.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955564692000003, 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.071 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:56.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955551100000001, 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:14:56.076 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:14:56.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:56.081 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955526818000003, 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955492657000001, 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:14:56.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.091 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955459259999998, 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:14:56.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:14:56.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955436614999996, 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:56.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:14:56.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955421039999997, 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:56.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.955404006000002, 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:14:56.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:14:56.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.113 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.951771041999997, 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:14:56.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.118 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.951736699999998, 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:14:56.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:14:56.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.951720754, 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:14:56.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:14:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.129 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.951705142999998, 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:14:56.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:14:56.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.134 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.951690055, 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.135 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:14:56.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.951675252999998, 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:14:56.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:14:56.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.145 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:56.146 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:56.147 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:56.145+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:56.247 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:56.243+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:56.254 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:56.254 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:56.254 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:14:56.255 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:56.264 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:56.349 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:56.341+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:56.361 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:56.361 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:56.361 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:56.361 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:56.361 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:56.372 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:56.449 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:56.445+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:56.494 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:56.494 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:56.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:56.494 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:56.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.494 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.357324195000004, 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:56.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.357305807000003, 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:56.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.357005049999998, 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:56.503 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:56.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.356975344999995, 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.508 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:56.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:56.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:56.512 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.356961947000002, 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:56.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.356948142999997, 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:56.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.521 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.521 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:56.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.356573143999995, 2024-09-16T11:14:56.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:56.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:56.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.343029113, 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:56.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:56.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.343007538999998, 2024-09-16T11:14:56.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.342991040000001, 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:56.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.342974964000007, 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:56.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.342591389999995, 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:56.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:56.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:56.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.330509212999999, 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:56.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.330501011999999, 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:56.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.330492989999996, 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.558 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:56.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.562 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 39.647014110999997, 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:14:56.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:14:56.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:56.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:56.568 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.41359576, 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:56.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:14:56.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.027202386999999, 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:14:56.574 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:14:56.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:14:56.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.075597738999999, 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:14:56.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:14:56.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 15.939800806999999, 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:14:56.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:14:56.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:56.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:56.594 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:56.596 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:56.596 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone37 from 0x480e7ed (overlap 0x122b2ee) to 0x105a109 2024-09-16T11:14:56.597 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:56.589+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:56.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:14:56.696 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:56.691+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:56.713 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:56.714 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:56.714 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:56.714 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:56.725 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:56.725 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-16T11:14:56.798 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:56.793+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:56.828 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:56.829 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:56.829 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:56.829 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:56.829 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:56.839 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:56.840 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:56.868 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:56.894 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:56.899 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:56.893+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:56.921 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:56.944 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:56.946 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:56.946 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:56.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:14:56.964 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:56.985 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:56.999 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:56.995+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:57.015 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:57.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:57.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:57.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:57.018 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:57.018 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:57.018 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:57.018 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:57.018 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:57.100 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:57.095+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:57.109 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.109 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.120 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:14:57.198 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.199 INFO:teuthology.orchestra.run.smithi162.stdout:4558 write 0x28c1f41 thru 0x28c754a (0x560a bytes) 2024-09-16T11:14:57.201 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:57.199+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:57.221 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.221 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.231 INFO:teuthology.orchestra.run.smithi162.stdout:4559 write 0x7c51dfd thru 0x7c5adf0 (0x8ff4 bytes) 2024-09-16T11:14:57.232 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:14:57.238 INFO:teuthology.orchestra.run.smithi162.stdout:4560 read 0x7af25c6 thru 0x7aff151 (0xcb8c bytes) 2024-09-16T11:14:57.238 INFO:teuthology.orchestra.run.smithi162.stdout:4561 read 0x157ab2c thru 0x1580b26 (0x5ffb bytes) 2024-09-16T11:14:57.238 INFO:teuthology.orchestra.run.smithi162.stdout:4562 write 0x6fe0dae thru 0x6fef104 (0xe357 bytes) 2024-09-16T11:14:57.278 INFO:teuthology.orchestra.run.smithi162.stdout:4563 write 0xedad96c thru 0xedb719a (0x982f bytes) 2024-09-16T11:14:57.286 INFO:teuthology.orchestra.run.smithi162.stdout:4564 write 0xd2677b4 thru 0xd26b0e3 (0x3930 bytes) 2024-09-16T11:14:57.293 INFO:teuthology.orchestra.run.smithi162.stdout:4565 read 0x9966e74 thru 0x996ab78 (0x3d05 bytes) 2024-09-16T11:14:57.294 INFO:teuthology.orchestra.run.smithi162.stdout:4566 write 0x73921c3 thru 0x7395613 (0x3451 bytes) 2024-09-16T11:14:57.298 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.298 INFO:teuthology.orchestra.run.smithi162.stdout:4569 punch from 0x8514a4 to 0x853416, (0x1f72 bytes) 2024-09-16T11:14:57.301 INFO:teuthology.orchestra.run.smithi162.stdout:4570 trunc from 0xedb719b to 0xe12e28b 2024-09-16T11:14:57.302 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:57.299+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:57.328 INFO:teuthology.orchestra.run.smithi162.stdout:4571 trunc from 0xe12e28b to 0x8f08d58 2024-09-16T11:14:57.330 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.330 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.340 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:14:57.355 INFO:teuthology.orchestra.run.smithi162.stdout:4574 trunc from 0x8f08d58 to 0xdd7f62e 2024-09-16T11:14:57.365 INFO:teuthology.orchestra.run.smithi162.stdout:4575 writesame 0x43bafdd thru 0x43bed59 data_size 0x21(0x3d7d bytes) 2024-09-16T11:14:57.372 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.372 INFO:teuthology.orchestra.run.smithi162.stdout:truncating to largest ever: 0xee3b211 2024-09-16T11:14:57.374 INFO:teuthology.orchestra.run.smithi162.stdout:4578 trunc from 0xdd7f62e to 0xee3b211 2024-09-16T11:14:57.385 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.385 INFO:teuthology.orchestra.run.smithi162.stdout:4583 write 0x3c7e949 thru 0x3c82962 (0x401a bytes) 2024-09-16T11:14:57.398 INFO:teuthology.orchestra.run.smithi162.stdout:4584 punch from 0xba5cd75 to 0xba65c04, (0x8e8f bytes) 2024-09-16T11:14:57.398 INFO:teuthology.orchestra.run.smithi162.stdout:4585 writesame 0xb1d00dd thru 0xb1d45fc data_size 0xe(0x4520 bytes) 2024-09-16T11:14:57.403 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:57.399+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:57.404 INFO:teuthology.orchestra.run.smithi162.stdout:4586 write 0xccc36a0 thru 0xccc4c16 (0x1577 bytes) 2024-09-16T11:14:57.409 INFO:teuthology.orchestra.run.smithi162.stdout:4588 read 0x672b1f4 thru 0x673919e (0xdfab bytes) 2024-09-16T11:14:57.410 INFO:teuthology.orchestra.run.smithi162.stdout:4591 writesame 0x9f62cc0 thru 0x9f6f428 data_size 0x1(0xc769 bytes) 2024-09-16T11:14:57.434 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.434 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.437 INFO:teuthology.orchestra.run.smithi162.stdout:4593 read 0x674e5a3 thru 0x6751e5d (0x38bb bytes) 2024-09-16T11:14:57.438 INFO:teuthology.orchestra.run.smithi162.stdout:4598 write 0x2fd74ab thru 0x2fe6d19 (0xf86f bytes) 2024-09-16T11:14:57.445 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:14:57.446 INFO:teuthology.orchestra.run.smithi162.stdout:4599 writesame 0x57ae382 thru 0x57b8e35 data_size 0x73(0xaab4 bytes) 2024-09-16T11:14:57.450 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:14:57.466 INFO:teuthology.orchestra.run.smithi162.stdout:4601 trunc from 0xee3b211 to 0xa79af0c 2024-09-16T11:14:57.486 INFO:teuthology.orchestra.run.smithi162.stdout:4603 punch from 0x4940ea5 to 0x49491f6, (0x8351 bytes) 2024-09-16T11:14:57.486 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.486 INFO:teuthology.orchestra.run.smithi162.stdout:4605 read 0x8c70dcb thru 0x8c75902 (0x4b38 bytes) 2024-09-16T11:14:57.486 INFO:teuthology.orchestra.run.smithi162.stdout:4606 punch from 0x1843dac to 0x1844339, (0x58d bytes) 2024-09-16T11:14:57.492 INFO:teuthology.orchestra.run.smithi162.stdout:4607 trunc from 0xa79af0c to 0x77a6e2a 2024-09-16T11:14:57.503 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:57.497+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:57.532 INFO:teuthology.orchestra.run.smithi162.stdout:4608 write 0xae9716b thru 0xaea154c (0xa3e2 bytes) 2024-09-16T11:14:57.545 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.545 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.546 INFO:teuthology.orchestra.run.smithi162.stdout:4609 punch from 0xbc9a1e to 0xbd7557, (0xdb39 bytes) 2024-09-16T11:14:57.546 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.547 INFO:teuthology.orchestra.run.smithi162.stdout:4612 read 0x70b14d4 thru 0x70c007b (0xeba8 bytes) 2024-09-16T11:14:57.547 INFO:teuthology.orchestra.run.smithi162.stdout:4613 punch from 0x8a934fa to 0x8aa26dc, (0xf1e2 bytes) 2024-09-16T11:14:57.547 INFO:teuthology.orchestra.run.smithi162.stdout:4615 punch from 0x6dc8e52 to 0x6dd15a6, (0x8754 bytes) 2024-09-16T11:14:57.547 INFO:teuthology.orchestra.run.smithi162.stdout:4617 write 0x8dfad0f thru 0x8e0567f (0xa971 bytes) 2024-09-16T11:14:57.552 INFO:teuthology.orchestra.run.smithi162.stdout:4618 writesame 0x30ff16e thru 0x310a455 data_size 0x64(0xb2e8 bytes) 2024-09-16T11:14:57.556 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:14:57.562 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.562 INFO:teuthology.orchestra.run.smithi162.stdout:4620 read 0x3583625 thru 0x3592e02 (0xf7de bytes) 2024-09-16T11:14:57.563 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.563 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:57.563 INFO:teuthology.orchestra.run.smithi162.stdout:4623 read 0x516cc89 thru 0x517b7fd (0xeb75 bytes) 2024-09-16T11:14:57.564 INFO:teuthology.orchestra.run.smithi162.stdout:4624 trunc from 0xaea154d to 0xa6e8b2f 2024-09-16T11:14:57.591 INFO:teuthology.orchestra.run.smithi162.stdout:4627 write 0xa6af2b9 thru 0xa6b74a1 (0x81e9 bytes) 2024-09-16T11:14:57.600 INFO:teuthology.orchestra.run.smithi162.stdout:4629 punch from 0x417859d to 0x4180ce1, (0x8744 bytes) 2024-09-16T11:14:57.600 INFO:teuthology.orchestra.run.smithi162.stdout:4631 read 0x727497b thru 0x727cd51 (0x83d7 bytes) 2024-09-16T11:14:57.600 INFO:teuthology.orchestra.run.smithi162.stdout:4632 writesame 0xa5a8395 thru 0xa5b28ac data_size 0x24(0xa518 bytes) 2024-09-16T11:14:57.604 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:57.597+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:57.608 INFO:teuthology.orchestra.run.smithi162.stdout:4634 punch from 0x7554cfe to 0x7559033, (0x4335 bytes) 2024-09-16T11:14:57.628 INFO:teuthology.orchestra.run.smithi162.stdout:4635 write 0xe30eeb7 thru 0xe31cbaa (0xdcf4 bytes) 2024-09-16T11:14:57.640 INFO:teuthology.orchestra.run.smithi162.stdout:4636 punch from 0x585145c to 0x58588f0, (0x7494 bytes) 2024-09-16T11:14:57.640 INFO:teuthology.orchestra.run.smithi162.stdout:4637 read 0x13d2995 thru 0x13e08fe (0xdf6a bytes) 2024-09-16T11:14:57.642 INFO:teuthology.orchestra.run.smithi162.stdout:4638 trunc from 0xe31cbab to 0xd38fab0 2024-09-16T11:14:57.660 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.660 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.670 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:14:57.705 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:57.703+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:57.762 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.762 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:14:57.806 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:57.801+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:57.870 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:57.870 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:57.879 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:14:57.925 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:57.917+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:58.007 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:58.001+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:58.027 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.027 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:14:58.108 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:58.103+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:58.145 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.145 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.155 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:14:58.209 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:58.201+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:58.249 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.250 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.258 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:14:58.309 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:58.305+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:58.353 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.354 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.363 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:14:58.413 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:58.409+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:58.460 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.460 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:14:58.514 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:58.509+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:58.567 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.567 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:14:58.621 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:58.619+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:58.675 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:14:58.683 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.684 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.687 INFO:teuthology.orchestra.run.smithi162.stdout:4639 read 0xab1cd4d thru 0xab29733 (0xc9e7 bytes) 2024-09-16T11:14:58.695 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:14:58.701 INFO:teuthology.orchestra.run.smithi162.stdout:4641 trunc from 0xd38fab0 to 0x4866011 2024-09-16T11:14:58.719 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:58.715+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:58.728 INFO:teuthology.orchestra.run.smithi162.stdout:4645 write 0x42c34cb thru 0x42c8cdf (0x5815 bytes) 2024-09-16T11:14:58.736 INFO:teuthology.orchestra.run.smithi162.stdout:4646 read 0x3580a06 thru 0x358b309 (0xa904 bytes) 2024-09-16T11:14:58.736 INFO:teuthology.orchestra.run.smithi162.stdout:4648 punch from 0x3e19da3 to 0x3e23f69, (0xa1c6 bytes) 2024-09-16T11:14:58.737 INFO:teuthology.orchestra.run.smithi162.stdout:4649 trunc from 0x4866011 to 0x4717189 2024-09-16T11:14:58.776 INFO:teuthology.orchestra.run.smithi162.stdout:4652 write 0xf24f16 thru 0xf2bcea (0x6dd5 bytes) 2024-09-16T11:14:58.798 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.798 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.807 INFO:teuthology.orchestra.run.smithi162.stdout:4655 trunc from 0x4717189 to 0xba6f50e 2024-09-16T11:14:58.811 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:14:58.821 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:14:58.813+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:14:58.825 INFO:teuthology.orchestra.run.smithi162.stdout:4657 trunc from 0xba6f50e to 0x869a31c 2024-09-16T11:14:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:4659 punch from 0x1ca7166 to 0x1cb0e2a, (0x9cc4 bytes) 2024-09-16T11:14:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:4663 read 0x4e6baa2 thru 0x4e7747e (0xb9dd bytes) 2024-09-16T11:14:58.838 INFO:teuthology.orchestra.run.smithi162.stdout:4665 read 0x4430bd0 thru 0x443bdd7 (0xb208 bytes) 2024-09-16T11:14:58.839 INFO:teuthology.orchestra.run.smithi162.stdout:4666 read 0x81f55ab thru 0x81fa4c9 (0x4f1f bytes) 2024-09-16T11:14:58.839 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.839 INFO:teuthology.orchestra.run.smithi162.stdout:4669 write 0x2565eee thru 0x25724c1 (0xc5d4 bytes) 2024-09-16T11:14:58.866 INFO:teuthology.orchestra.run.smithi162.stdout:4670 write 0x88e6642 thru 0x88e8d5b (0x271a bytes) 2024-09-16T11:14:58.893 INFO:teuthology.orchestra.run.smithi162.stdout:4671 write 0xaa09f85 thru 0xaa0a4f3 (0x56f bytes) 2024-09-16T11:14:58.899 INFO:teuthology.orchestra.run.smithi162.stdout:4672 punch from 0x7ce2130 to 0x7ce71ee, (0x50be bytes) 2024-09-16T11:14:58.902 INFO:teuthology.orchestra.run.smithi162.stdout:4673 trunc from 0xaa0a4f4 to 0xc4b4e13 2024-09-16T11:14:58.915 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.916 INFO:teuthology.orchestra.run.smithi162.stdout:4675 read 0x8d26d42 thru 0x8d34947 (0xdc06 bytes) 2024-09-16T11:14:58.916 INFO:teuthology.orchestra.run.smithi162.stdout:4676 write 0x4cb1b02 thru 0x4cb3182 (0x1681 bytes) 2024-09-16T11:14:58.922 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:58.917+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:58.923 INFO:teuthology.orchestra.run.smithi162.stdout:4677 punch from 0xa342870 to 0xa34c604, (0x9d94 bytes) 2024-09-16T11:14:58.923 INFO:teuthology.orchestra.run.smithi162.stdout:4678 read 0x6465111 thru 0x6472973 (0xd863 bytes) 2024-09-16T11:14:58.923 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.923 INFO:teuthology.orchestra.run.smithi162.stdout:4681 punch from 0x5a66750 to 0x5a6a77e, (0x402e bytes) 2024-09-16T11:14:58.923 INFO:teuthology.orchestra.run.smithi162.stdout:4682 punch from 0x2c07d2a to 0x2c0bb4c, (0x3e22 bytes) 2024-09-16T11:14:58.923 INFO:teuthology.orchestra.run.smithi162.stdout:4683 write 0x14cf00a thru 0x14d32fb (0x42f2 bytes) 2024-09-16T11:14:58.927 INFO:teuthology.orchestra.run.smithi162.stdout:4684 read 0x3121211 thru 0x3130211 (0xf001 bytes) 2024-09-16T11:14:58.928 INFO:teuthology.orchestra.run.smithi162.stdout:4685 trunc from 0xc4b4e13 to 0xce5698d 2024-09-16T11:14:58.930 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:58.931 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:58.941 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:14:58.946 INFO:teuthology.orchestra.run.smithi162.stdout:4686 read 0xa5fd162 thru 0xa605822 (0x86c1 bytes) 2024-09-16T11:14:58.947 INFO:teuthology.orchestra.run.smithi162.stdout:4687 write 0x9922d thru 0xa65f1 (0xd3c5 bytes) 2024-09-16T11:14:58.973 INFO:teuthology.orchestra.run.smithi162.stdout:4688 read 0x9162235 thru 0x9168eec (0x6cb8 bytes) 2024-09-16T11:14:58.974 INFO:teuthology.orchestra.run.smithi162.stdout:4689 read 0x66a8528 thru 0x66a9aab (0x1584 bytes) 2024-09-16T11:14:58.974 INFO:teuthology.orchestra.run.smithi162.stdout:4690 writesame 0x790ea7d thru 0x7918190 data_size 0x42(0x9714 bytes) 2024-09-16T11:14:58.984 INFO:teuthology.orchestra.run.smithi162.stdout:4691 write 0x68acc3c thru 0x68b1dc0 (0x5185 bytes) 2024-09-16T11:14:58.992 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:58.995 INFO:teuthology.orchestra.run.smithi162.stdout:4693 trunc from 0xce5698d to 0xb6f180a 2024-09-16T11:14:59.006 INFO:teuthology.orchestra.run.smithi162.stdout:4694 write 0x75a6cec thru 0x75ac147 (0x545c bytes) 2024-09-16T11:14:59.014 INFO:teuthology.orchestra.run.smithi162.stdout:4695 read 0x193d1ed thru 0x1945f3d (0x8d51 bytes) 2024-09-16T11:14:59.025 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:59.021+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:59.032 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:14:59.033 INFO:teuthology.orchestra.run.smithi162.stdout:4696 writesame 0xbc062d7 thru 0xbc12740 data_size 0x1f(0xc46a bytes) 2024-09-16T11:14:59.036 INFO:teuthology.orchestra.run.smithi162.stdout:4697 trunc from 0xbc12741 to 0x9ec2063 2024-09-16T11:14:59.058 INFO:teuthology.orchestra.run.smithi162.stdout:4699 read 0x61279aa thru 0x61376fd (0xfd54 bytes) 2024-09-16T11:14:59.058 INFO:teuthology.orchestra.run.smithi162.stdout:4700 write 0x3ba0d0b thru 0x3babc19 (0xaf0f bytes) 2024-09-16T11:14:59.060 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:59.061 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:59.066 INFO:teuthology.orchestra.run.smithi162.stdout:4701 read 0x20ff539 thru 0x210ce75 (0xd93d bytes) 2024-09-16T11:14:59.066 INFO:teuthology.orchestra.run.smithi162.stdout:4702 read 0x12ff63d thru 0x1309760 (0xa124 bytes) 2024-09-16T11:14:59.073 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:14:59.093 INFO:teuthology.orchestra.run.smithi162.stdout:4703 write 0xcccab4a thru 0xccd64c4 (0xb97b bytes) 2024-09-16T11:14:59.101 INFO:teuthology.orchestra.run.smithi162.stdout:4704 read 0x9f0d913 thru 0x9f14998 (0x7086 bytes) 2024-09-16T11:14:59.101 INFO:teuthology.orchestra.run.smithi162.stdout:4705 punch from 0xba35cf to 0xbb06a6, (0xd0d7 bytes) 2024-09-16T11:14:59.101 INFO:teuthology.orchestra.run.smithi162.stdout:4706 write 0x18066e1 thru 0x1811961 (0xb281 bytes) 2024-09-16T11:14:59.113 INFO:teuthology.orchestra.run.smithi162.stdout:4707 write 0x2c2bd20 thru 0x2c3a1f8 (0xe4d9 bytes) 2024-09-16T11:14:59.122 INFO:teuthology.orchestra.run.smithi162.stdout:4708 read 0xc431248 thru 0xc43fedf (0xec98 bytes) 2024-09-16T11:14:59.123 INFO:teuthology.orchestra.run.smithi162.stdout:4709 write 0xaf0ebf thru 0xaf0f1d (0x5f bytes) 2024-09-16T11:14:59.128 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:14:59.123+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:4710 punch from 0xc37de4 to 0xc46166, (0xe382 bytes) 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:4713 read 0x217cd82 thru 0x217dd4c (0xfcb bytes) 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:4719 punch from 0x1015de9 to 0x10216b8, (0xb8cf bytes) 2024-09-16T11:14:59.141 INFO:teuthology.orchestra.run.smithi162.stdout:4720 write 0xc7190d4 thru 0xc724241 (0xb16e bytes) 2024-09-16T11:14:59.153 INFO:teuthology.orchestra.run.smithi162.stdout:4721 write 0x281529 thru 0x28d3a7 (0xbe7f bytes) 2024-09-16T11:14:59.178 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:14:59.181 INFO:teuthology.orchestra.run.smithi162.stdout:4722 writesame 0x4b9dfe6 thru 0x4ba07ab data_size 0x3(0x27c6 bytes) 2024-09-16T11:14:59.192 INFO:teuthology.orchestra.run.smithi162.stdout:4723 trunc from 0xccd64c5 to 0xc3cbe37 2024-09-16T11:14:59.199 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:14:59.206 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:14:59.206 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:14:59.214 INFO:teuthology.orchestra.run.smithi162.stdout:4724 read 0x3445f1f thru 0x34491f6 (0x32d8 bytes) 2024-09-16T11:14:59.214 INFO:teuthology.orchestra.run.smithi162.stdout:4725 writesame 0x1d922f5 thru 0x1d9dda3 data_size 0x1(0xbaaf bytes) 2024-09-16T11:14:59.215 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:14:59.217 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:14:59.236 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:14:59.237 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:14:59.235+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:14:59.246 INFO:teuthology.orchestra.run.smithi162.stdout:4726 writesame 0xc32daaa thru 0xc335b3e data_size 0x1(0x8095 bytes) 2024-09-16T11:14:59.259 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:14:59.269 INFO:teuthology.orchestra.run.smithi162.stdout:4727 read 0x9e0ca35 thru 0x9e13617 (0x6be3 bytes) 2024-09-16T11:14:59.270 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.280 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:14:59.285 INFO:teuthology.orchestra.run.smithi162.stdout:4729 write 0xdda13b6 thru 0xddabbaf (0xa7fa bytes) 2024-09-16T11:14:59.294 INFO:teuthology.orchestra.run.smithi162.stdout:4730 read 0x93da6b9 thru 0x93e44dd (0x9e25 bytes) 2024-09-16T11:14:59.294 INFO:teuthology.orchestra.run.smithi162.stdout:4732 punch from 0x2cc64f2 to 0x2cd3202, (0xcd10 bytes) 2024-09-16T11:14:59.294 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.303 INFO:teuthology.orchestra.run.smithi162.stdout:4737 trunc from 0xddabbb0 to 0x7f8f1ce 2024-09-16T11:14:59.310 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:14:59.324 INFO:teuthology.orchestra.run.smithi162.stdout:4739 write 0x77d5bd0 thru 0x77d9803 (0x3c34 bytes) 2024-09-16T11:14:59.325 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:59.325 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:14:59.325 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:14:59.325 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:14:59.325 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:59.327 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:14:59.323+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:14:59.336 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:14:59.336 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:14:59.338 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:14:59.338 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:14:59.338 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:14:59.339 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:14:59.339 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:14:59.339 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:14:59.339 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:14:59.339 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:14:59.339 INFO:teuthology.orchestra.run.smithi162.stdout:4741 trunc from 0x7f8f1ce to 0xbea446a 2024-09-16T11:14:59.353 INFO:teuthology.orchestra.run.smithi162.stdout:4742 trunc from 0xbea446a to 0x9513f73 2024-09-16T11:14:59.381 INFO:teuthology.orchestra.run.smithi162.stdout:4744 write 0xaf6812c thru 0xaf6ac29 (0x2afe bytes) 2024-09-16T11:14:59.402 INFO:teuthology.orchestra.run.smithi162.stdout:4746 writesame 0xb5a34c3 thru 0xb5a83a9 data_size 0x1(0x4ee7 bytes) 2024-09-16T11:14:59.402 INFO:teuthology.orchestra.run.smithi162.stdout:4747 write 0xb4d4ed2 thru 0xb4dc96b (0x7a9a bytes) 2024-09-16T11:14:59.428 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:59.421+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:59.429 INFO:teuthology.orchestra.run.smithi162.stdout:4748 trunc from 0xb5a83aa to 0x9fa39e 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.314115534999999, 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:14:59.451 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.455 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.314097146999998, 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:59.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.460 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.313796389999993, 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:14:59.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.313766685000004, 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:14:59.465 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.313753286999997, 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:14:59.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:14:59.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:14:59.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:14:59.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.313739483000006, 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:14:59.475 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.313364484000004, 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:59.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.299820452999995, 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.485 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:14:59.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:14:59.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.299798878999994, 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.299782379999996, 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.493 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:14:59.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:14:59.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.299766304000002, 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:14:59.498 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:14:59.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.299382730000005, 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:14:59.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:14:59.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:14:59.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.287300552999994, 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.507 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.287292351999994, 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:59.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:14:59.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.287284330000006, 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:59.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:14:59.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:14:59.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.603805450999999, 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:14:59.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.370387100000002, 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:14:59.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:14:59.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:14:59.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 27.983993727000001, 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:14:59.532 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:14:59.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:14:59.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:14:59.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 26.032389079000001, 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:14:59.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:14:59.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:14:59.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 18.896592147, 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:14:59.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:14:59.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:14:59.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:14:59.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:14:59.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:14:59.549 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:14:59.551 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:14:59.551 INFO:teuthology.orchestra.run.smithi162.stdout:4749 write 0xadd16fb thru 0xadd3288 (0x1b8e bytes) 2024-09-16T11:14:59.552 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:14:59.545+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:14:59.558 INFO:teuthology.orchestra.run.smithi162.stdout:4750 punch from 0x6803640 to 0x680ec3c, (0xb5fc bytes) 2024-09-16T11:14:59.558 INFO:teuthology.orchestra.run.smithi162.stdout:4751 read 0x86e0b5a thru 0x86e0e6c (0x313 bytes) 2024-09-16T11:14:59.558 INFO:teuthology.orchestra.run.smithi162.stdout:4755 punch from 0x390874e to 0x390f178, (0x6a2a bytes) 2024-09-16T11:14:59.572 INFO:teuthology.orchestra.run.smithi162.stdout:4756 trunc from 0xadd3289 to 0x3c0db7c 2024-09-16T11:14:59.593 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.611 INFO:teuthology.orchestra.run.smithi162.stdout:4758 writesame 0x6b5c4c3 thru 0x6b61d66 data_size 0x3e(0x58a4 bytes) 2024-09-16T11:14:59.611 INFO:teuthology.orchestra.run.smithi162.stdout:4761 read 0x3bfe0d7 thru 0x3c0da46 (0xf970 bytes) 2024-09-16T11:14:59.632 INFO:teuthology.orchestra.run.smithi162.stdout:4762 writesame 0x9ff2221 thru 0x9ff3d64 data_size 0xa(0x1b44 bytes) 2024-09-16T11:14:59.632 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.632 INFO:teuthology.orchestra.run.smithi162.stdout:4765 read 0x9e144d3 thru 0x9e23e29 (0xf957 bytes) 2024-09-16T11:14:59.632 INFO:teuthology.orchestra.run.smithi162.stdout:4767 writesame 0x41044bc thru 0x410bf04 data_size 0x5(0x7a49 bytes) 2024-09-16T11:14:59.646 INFO:teuthology.orchestra.run.smithi162.stdout:4768 trunc from 0x9ff3d65 to 0x9b494ea 2024-09-16T11:14:59.651 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:14:59.647+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:14:59.658 INFO:teuthology.orchestra.run.smithi162.stdout:4769 writesame 0x2a436c3 thru 0x2a44878 data_size 0x1(0x11b6 bytes) 2024-09-16T11:14:59.663 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:59.663 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:59.664 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:14:59.664 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:59.668 INFO:teuthology.orchestra.run.smithi162.stdout:4770 write 0x38a1650 thru 0x38aa785 (0x9136 bytes) 2024-09-16T11:14:59.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:14:59.678 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 6, 7, 2] out_osds: [4, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:14:59.678 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:14:59.678 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-16T11:14:59.678 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-16T11:14:59.687 INFO:teuthology.orchestra.run.smithi162.stdout:4771 trunc from 0x9b494ea to 0x54f96fa 2024-09-16T11:14:59.726 INFO:teuthology.orchestra.run.smithi162.stdout:4772 writesame 0xd33ec3b thru 0xd346df1 data_size 0x1(0x81b7 bytes) 2024-09-16T11:14:59.726 INFO:teuthology.orchestra.run.smithi162.stdout:4773 read 0xc1205b0 thru 0xc12ff64 (0xf9b5 bytes) 2024-09-16T11:14:59.726 INFO:teuthology.orchestra.run.smithi162.stdout:4774 read 0x3e9df16 thru 0x3eaa8dd (0xc9c8 bytes) 2024-09-16T11:14:59.726 INFO:teuthology.orchestra.run.smithi162.stdout:4777 write 0xa802dd4 thru 0xa80acc4 (0x7ef1 bytes) 2024-09-16T11:14:59.734 INFO:teuthology.orchestra.run.smithi162.stdout:4778 read 0x822c5a3 thru 0x8238379 (0xbdd7 bytes) 2024-09-16T11:14:59.734 INFO:teuthology.orchestra.run.smithi162.stdout:4779 write 0x41c22a1 thru 0x41cc79b (0xa4fb bytes) 2024-09-16T11:14:59.737 INFO:teuthology.orchestra.run.smithi162.stdout:4781 writesame 0xce24edd thru 0xce3467c data_size 0xe(0xf7a0 bytes) 2024-09-16T11:14:59.751 INFO:teuthology.orchestra.run.smithi162.stdout:4782 punch from 0x322ed61 to 0x323b146, (0xc3e5 bytes) 2024-09-16T11:14:59.751 INFO:teuthology.orchestra.run.smithi162.stdout:4783 read 0xb3bb651 thru 0xb3c1e94 (0x6844 bytes) 2024-09-16T11:14:59.751 INFO:teuthology.orchestra.run.smithi162.stdout:4784 read 0x85b96be thru 0x85bdf44 (0x4887 bytes) 2024-09-16T11:14:59.751 INFO:teuthology.orchestra.run.smithi162.stdout:4785 write 0x27d6af6 thru 0x27db862 (0x4d6d bytes) 2024-09-16T11:14:59.752 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:14:59.745+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:14:59.759 INFO:teuthology.orchestra.run.smithi162.stdout:4786 writesame 0x5d80956 thru 0x5d870a4 data_size 0x1(0x674f bytes) 2024-09-16T11:14:59.774 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:59.774 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:14:59.775 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:14:59.775 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:14:59.775 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:14:59.784 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:14:59.785 INFO:teuthology.orchestra.run.smithi162.stdout:4788 writesame 0x15b2a40 thru 0x15b63c0 data_size 0x7(0x3981 bytes) 2024-09-16T11:14:59.791 INFO:teuthology.orchestra.run.smithi162.stdout:4789 write 0x45a4700 thru 0x45a4791 (0x92 bytes) 2024-09-16T11:14:59.797 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.797 INFO:teuthology.orchestra.run.smithi162.stdout:4792 punch from 0xcdd2fa6 to 0xcddf065, (0xc0bf bytes) 2024-09-16T11:14:59.797 INFO:teuthology.orchestra.run.smithi162.stdout:4793 read 0xbe773ca thru 0xbe7add2 (0x3a09 bytes) 2024-09-16T11:14:59.797 INFO:teuthology.orchestra.run.smithi162.stdout:4794 read 0x55a461c thru 0x55aef10 (0xa8f5 bytes) 2024-09-16T11:14:59.798 INFO:teuthology.orchestra.run.smithi162.stdout:4796 punch from 0x93fb49e to 0x93fcb5b, (0x16bd bytes) 2024-09-16T11:14:59.798 INFO:teuthology.orchestra.run.smithi162.stdout:4797 punch from 0x2aaca44 to 0x2aaca89, (0x45 bytes) 2024-09-16T11:14:59.798 INFO:teuthology.orchestra.run.smithi162.stdout:4798 write 0x1d09722 thru 0x1d17fab (0xe88a bytes) 2024-09-16T11:14:59.804 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:4802 read 0x692b216 thru 0x6936618 (0xb403 bytes) 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:4804 read 0x8cf12e1 thru 0x8cf9024 (0x7d44 bytes) 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:4807 read 0x2f3e06f thru 0x2f42b55 (0x4ae7 bytes) 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.805 INFO:teuthology.orchestra.run.smithi162.stdout:4811 write 0x8f105f1 thru 0x8f18ef6 (0x8906 bytes) 2024-09-16T11:14:59.810 INFO:teuthology.orchestra.run.smithi162.stdout:4812 write 0x88b7de3 thru 0x88c471a (0xc938 bytes) 2024-09-16T11:14:59.818 INFO:teuthology.orchestra.run.smithi162.stdout:4813 write 0x309a80d thru 0x30a61ef (0xb9e3 bytes) 2024-09-16T11:14:59.825 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.826 INFO:teuthology.orchestra.run.smithi162.stdout:4815 read 0x184c826 thru 0x184d31b (0xaf6 bytes) 2024-09-16T11:14:59.826 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.826 INFO:teuthology.orchestra.run.smithi162.stdout:4818 read 0xb9758b0 thru 0xb977c21 (0x2372 bytes) 2024-09-16T11:14:59.826 INFO:teuthology.orchestra.run.smithi162.stdout:4819 read 0x1ac4d0b thru 0x1ac5691 (0x987 bytes) 2024-09-16T11:14:59.826 INFO:teuthology.orchestra.run.smithi162.stdout:4820 read 0x9aca3c3 thru 0x9ad22f3 (0x7f31 bytes) 2024-09-16T11:14:59.826 INFO:teuthology.orchestra.run.smithi162.stdout:4821 write 0x7d5e882 thru 0x7d6dbc2 (0xf341 bytes) 2024-09-16T11:14:59.835 INFO:teuthology.orchestra.run.smithi162.stdout:4823 writesame 0x3dc8297 thru 0x3dd5087 data_size 0x1(0xcdf1 bytes) 2024-09-16T11:14:59.852 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:14:59.845+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:14:59.860 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:14:59.860 INFO:teuthology.orchestra.run.smithi162.stdout:4825 punch from 0x73c63b to 0x741f22, (0x58e7 bytes) 2024-09-16T11:14:59.860 INFO:teuthology.orchestra.run.smithi162.stdout:4828 write 0xa5a378e thru 0xa5a86dc (0x4f4f bytes) 2024-09-16T11:14:59.885 INFO:teuthology.orchestra.run.smithi162.stdout:4829 trunc from 0xd346df2 to 0x2655742 2024-09-16T11:14:59.896 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819483640000001, 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:14:59.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.904 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819464273999998, 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:14:59.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:14:59.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819423512, 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.910 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:14:59.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819392714000003, 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:14:59.916 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:14:59.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:14:59.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819374126, 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:14:59.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819356073999998, 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:14:59.927 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:59.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819336147000001, 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:14:59.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:14:59.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:59.939 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819317308000002, 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:14:59.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:14:59.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819125358999997, 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.946 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.951 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.819106079000001, 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:14:59.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:14:59.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815635813999997, 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:14:59.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:14:59.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:59.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:14:59.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.963 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:59.964 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815614511, 2024-09-16T11:14:59.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:14:59.964 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.964 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.965 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.965 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:14:59.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.965 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:59.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:14:59.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:14:59.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:14:59.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:14:59.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:14:59.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:14:59.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:14:59.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:14:59.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:14:59.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:14:59.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:14:59.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:14:59.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:14:59.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:14:59.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:14:59.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:14:59.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:14:59.974 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.974 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.975 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.975 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:59.976 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815604440000001, 2024-09-16T11:14:59.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:14:59.976 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.976 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.977 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:14:59.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.977 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:59.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:14:59.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:14:59.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:14:59.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:14:59.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815576137000001, 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.984 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:59.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:14:59.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:14:59.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815562208000003, 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:14:59.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.994 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815553444999999, 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.995 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:14:59.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:14:59.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:00.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:00.001 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815529626, 2024-09-16T11:15:00.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:00.002 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.002 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:00.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:00.008 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815498503999997, 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:00.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:00.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815450765000001, 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:00.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:00.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.815428656999998, 2024-09-16T11:15:00.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:00.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.026 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.026 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:00.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:4832 read 0x94ac80 thru 0x94b330 (0x6b1 bytes) 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:4834 punch from 0x80c877 to 0x80fa67, (0x31f0 bytes) 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:4837 trunc from 0x2655742 to 0xd229dfc 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:4838 read 0xa0730a4 thru 0xa079c76 (0x6bd3 bytes) 2024-09-16T11:15:00.029 INFO:teuthology.orchestra.run.smithi162.stdout:4840 read 0xce427f6 thru 0xce4954b (0x6d56 bytes) 2024-09-16T11:15:00.030 INFO:teuthology.orchestra.run.smithi162.stdout:4841 trunc from 0xd229dfc to 0x6357f3c 2024-09-16T11:15:00.030 INFO:teuthology.orchestra.run.smithi162.stdout:4842 read 0x2964c24 thru 0x2966cfc (0x20d9 bytes) 2024-09-16T11:15:00.030 INFO:teuthology.orchestra.run.smithi162.stdout:4843 read 0x20f75a0 thru 0x21048c7 (0xd328 bytes) 2024-09-16T11:15:00.030 INFO:teuthology.orchestra.run.smithi162.stdout:4844 read 0x430f9a2 thru 0x4312e95 (0x34f4 bytes) 2024-09-16T11:15:00.030 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.030 INFO:teuthology.orchestra.run.smithi162.stdout:4846 trunc from 0x6357f3c to 0x8cc4158 2024-09-16T11:15:00.031 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:14:59.991+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e114 definitely_dead 0 2024-09-16T11:15:00.033 INFO:teuthology.orchestra.run.smithi162.stdout:4847 write 0xb7024ab thru 0xb709cef (0x7845 bytes) 2024-09-16T11:15:00.033 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.033 INFO:teuthology.orchestra.run.smithi162.stdout:4850 write 0xb1c1127 thru 0xb1c47ac (0x3686 bytes) 2024-09-16T11:15:00.033 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.034 INFO:teuthology.orchestra.run.smithi162.stdout:4853 read 0x25384a9 thru 0x25438d6 (0xb42e bytes) 2024-09-16T11:15:00.034 INFO:teuthology.orchestra.run.smithi162.stdout:4854 writesame 0x1451469 thru 0x1451628 data_size 0xe(0x1c0 bytes) 2024-09-16T11:15:00.034 INFO:teuthology.orchestra.run.smithi162.stdout:4855 read 0xac38cec thru 0xac3ced8 (0x41ed bytes) 2024-09-16T11:15:00.034 INFO:teuthology.orchestra.run.smithi162.stdout:4856 write 0x2e539da thru 0x2e5ff9b (0xc5c2 bytes) 2024-09-16T11:15:00.035 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:00.023+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:00.036 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.036 INFO:teuthology.orchestra.run.smithi162.stdout:4859 read 0xad1009a thru 0xad12032 (0x1f99 bytes) 2024-09-16T11:15:00.037 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.037 INFO:teuthology.orchestra.run.smithi162.stdout:4861 read 0xaafcdd6 thru 0xab0573e (0x8969 bytes) 2024-09-16T11:15:00.037 INFO:teuthology.orchestra.run.smithi162.stdout:4863 read 0x3ba833c thru 0x3bb579e (0xd463 bytes) 2024-09-16T11:15:00.037 INFO:teuthology.orchestra.run.smithi162.stdout:4864 read 0xb0ed4ce thru 0xb0ef3f4 (0x1f27 bytes) 2024-09-16T11:15:00.037 INFO:teuthology.orchestra.run.smithi162.stdout:4865 read 0xa9c5e71 thru 0xa9ce656 (0x87e6 bytes) 2024-09-16T11:15:00.037 INFO:teuthology.orchestra.run.smithi162.stdout:4867 read 0x33932ab thru 0x33a0338 (0xd08e bytes) 2024-09-16T11:15:00.038 INFO:teuthology.orchestra.run.smithi162.stdout:4868 read 0x5b8a8ca thru 0x5b99030 (0xe767 bytes) 2024-09-16T11:15:00.038 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.052 INFO:teuthology.orchestra.run.smithi162.stdout:4873 trunc from 0xb709cf0 to 0x1e2de28 2024-09-16T11:15:00.094 INFO:teuthology.orchestra.run.smithi162.stdout:4874 writesame 0x819f647 thru 0x81a1810 data_size 0x19(0x21ca bytes) 2024-09-16T11:15:00.114 INFO:teuthology.orchestra.run.smithi162.stdout:4875 write 0xd5e1f64 thru 0xd5e42ad (0x234a bytes) 2024-09-16T11:15:00.120 INFO:teuthology.orchestra.run.smithi162.stdout:4876 read 0x442945d thru 0x442c905 (0x34a9 bytes) 2024-09-16T11:15:00.120 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.128 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:00.121+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:00.139 INFO:teuthology.orchestra.run.smithi162.stdout:4878 trunc from 0xd5e42ae to 0x27018f1 2024-09-16T11:15:00.151 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:00.151 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:00.151 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:00.151 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:00.159 INFO:teuthology.orchestra.run.smithi162.stdout:4879 punch from 0x1ae39d to 0x1b5563, (0x71c6 bytes) 2024-09-16T11:15:00.159 INFO:teuthology.orchestra.run.smithi162.stdout:4880 punch from 0x19c0ad0 to 0x19c0e76, (0x3a6 bytes) 2024-09-16T11:15:00.161 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:00.178 INFO:teuthology.orchestra.run.smithi162.stdout:4881 writesame 0x7c1a484 thru 0x7c1fa01 data_size 0x1f(0x557e bytes) 2024-09-16T11:15:00.178 INFO:teuthology.orchestra.run.smithi162.stdout:4882 punch from 0x60da74a to 0x60deefd, (0x47b3 bytes) 2024-09-16T11:15:00.194 INFO:teuthology.orchestra.run.smithi162.stdout:4883 write 0xb697e0d thru 0xb6a0338 (0x852c bytes) 2024-09-16T11:15:00.217 INFO:teuthology.orchestra.run.smithi162.stdout:4884 writesame 0xc732f86 thru 0xc738f0d data_size 0xc(0x5f88 bytes) 2024-09-16T11:15:00.218 INFO:teuthology.orchestra.run.smithi162.stdout:4885 read 0x612cc7a thru 0x6131552 (0x48d9 bytes) 2024-09-16T11:15:00.218 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:00.218 INFO:teuthology.orchestra.run.smithi162.stdout:4888 write 0x729f7ed thru 0x72aebde (0xf3f2 bytes) 2024-09-16T11:15:00.226 INFO:teuthology.orchestra.run.smithi162.stdout:4889 punch from 0x8e2edee to 0x8e39960, (0xab72 bytes) 2024-09-16T11:15:00.226 INFO:teuthology.orchestra.run.smithi162.stdout:4890 write 0x2e65d3d thru 0x2e7540d (0xf6d1 bytes) 2024-09-16T11:15:00.229 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:00.221+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:00.234 INFO:teuthology.orchestra.run.smithi162.stdout:4891 read 0x2f6a907 thru 0x2f6fccb (0x53c5 bytes) 2024-09-16T11:15:00.234 INFO:teuthology.orchestra.run.smithi162.stdout:4892 clone 39 order 18 su 131072 sc 2 2024-09-16T11:15:00.260 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:00.261 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:00.261 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:00.261 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:00.261 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:00.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:00.329 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:00.325+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.547820420999997, 2024-09-16T11:15:00.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:00.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379848772000003, 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.420 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:00.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379837210999995, 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:00.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:00.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.430 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379824009000004, 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:00.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.435 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379811568999997, 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:00.436 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.437 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.437 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:00.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:00.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:00.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:00.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:00.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:00.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:00.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379798058000006, 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.445 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:00.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379787143000001, 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:00.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:00.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:00.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:00.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:00.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:00.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:00.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:00.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:00.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:00.457 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.457 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.457 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.457 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.458 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:00.458 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379761224999996, 2024-09-16T11:15:00.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:00.458 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.458 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.459 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:00.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.459 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:00.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:00.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:00.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:00.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:00.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:00.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379745771000003, 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:00.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:00.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.473 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379735885000002, 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:00.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:00.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:00.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:00.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.479 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.479 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:00.479 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379726121999994, 2024-09-16T11:15:00.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:00.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:00.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379715945000001, 2024-09-16T11:15:00.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:00.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:00.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:00.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:00.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379699462999994, 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:00.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:00.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:00.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379684765999997, 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.496 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:00.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:00.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379666322999995, 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.502 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:00.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.506 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379649373999996, 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:00.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:00.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:00.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:00.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:00.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.511 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.511 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:00.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:00.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:00.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:00.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:00.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:00.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.514 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.514 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:00.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379630160999994, 2024-09-16T11:15:00.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:00.516 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.516 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.517 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:00.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.517 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:00.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:00.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:00.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:00.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379612945000005, 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:00.522 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:00.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:00.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379591691000002, 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:00.528 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:00.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:00.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:00.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.379571722999998, 2024-09-16T11:15:00.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:00.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:00.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:00.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:00.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:00.540 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.540 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.541 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.541 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.541 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.541 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:00.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:00.544 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:00.537+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:00.644 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:00.639+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:00.681 INFO:teuthology.orchestra.run.smithi162.stderr:marked in osd.4. 2024-09-16T11:15:00.685 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:15:00.692 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-16T11:15:00.695 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:00.695 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:00.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.695 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:168097 4.5 4.c18068c5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T11:15:00.695 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:00.665838+0000", 2024-09-16T11:15:00.695 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.028577340999999999, 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.028600607, 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 168097 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.665838+0000", 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.665838+0000", 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.665841+0000", 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6229999999999999e-06 2024-09-16T11:15:00.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.665856+0000", 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5173e-05 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.665900+0000", 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3482999999999997e-05 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.665915+0000", 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5556000000000001e-05 2024-09-16T11:15:00.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.666082+0000", 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016627800000000001 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op must wait for map", 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.666100+0000", 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8009999999999999e-05 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.672365+0000", 2024-09-16T11:15:00.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062655790000000003 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.672390+0000", 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5123e-05 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:168098 4.5 4.c18068c5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:00.671394+0000", 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.023020974999999999, 2024-09-16T11:15:00.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023112088999999999, 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 168098 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.671394+0000", 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.671394+0000", 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.671397+0000", 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.65e-06 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.671411+0000", 2024-09-16T11:15:00.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4039000000000001e-05 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.671441+0000", 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.006e-05 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.671458+0000", 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7133999999999999e-05 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.672465+0000", 2024-09-16T11:15:00.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001007172 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:00.672487+0000", 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2467e-05 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.705 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:15:00.706 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 2 2024-09-16T11:15:00.706 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:00.707 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:00.744 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:00.739+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:00.800 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:00.801 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:00.801 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:00.801 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:00.801 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:00.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:00.845 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:00.843+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:00.914 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:00.914 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:00.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50165 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1032192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.725268259000003, 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118612279, 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50165 2024-09-16T11:15:00.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.185784+0000", 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189847+0000", 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0040633300000000004 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189854+0000", 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8750000000000002e-06 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189874+0000", 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0242999999999999e-05 2024-09-16T11:15:00.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.189882+0000", 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6939999999999995e-06 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.303898+0000", 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11401576099999999 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304029+0000", 2024-09-16T11:15:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000131579 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.304396+0000", 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036679699999999998 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50187 4.1f 4:fafe5d12:::rbd_data.11b5eaebe1b2.0000000000000008:11 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:15:00.919 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.720909731999996, 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118815584, 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50187 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190142+0000", 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190143+0000", 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190145+0000", 2024-09-16T11:15:00.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5179999999999999e-06 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190152+0000", 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0219999999999999e-06 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.190157+0000", 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1609999999999996e-06 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308457+0000", 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118300138 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308581+0000", 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124375 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.308958+0000", 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00037687400000000002 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50302 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 450560~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.717271202000006, 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037741, 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50302 2024-09-16T11:15:00.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193781+0000", 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.58e-07 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193784+0000", 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.249e-06 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193790+0000", 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7920000000000004e-06 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193795+0000", 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312271+0000", 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11847645399999999 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312407+0000", 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013569799999999999 2024-09-16T11:15:00.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312819+0000", 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041170300000000003 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50303 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.717252415999994, 2024-09-16T11:15:00.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119477392, 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50303 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193800+0000", 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5200000000000002e-07 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193802+0000", 2024-09-16T11:15:00.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193809+0000", 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4309999999999999e-06 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193813+0000", 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7519999999999998e-06 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312850+0000", 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119037745 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.312964+0000", 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113825 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313277+0000", 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00031306599999999997 2024-09-16T11:15:00.932 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50304 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.717235092999999, 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120059121, 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50304 2024-09-16T11:15:00.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193817+0000", 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193818+0000", 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2899999999999999e-07 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193820+0000", 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193826+0000", 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5749999999999997e-06 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193830+0000", 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8310000000000003e-06 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313332+0000", 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11950203399999999 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313486+0000", 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015432 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313876+0000", 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00038977199999999998 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.717217624, 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:15:00.937 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.941 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50308 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 843776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713377164999997, 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11856446, 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50308 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197675+0000", 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2399999999999999e-07 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197677+0000", 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.367e-06 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197680+0000", 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197683+0000", 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5160000000000001e-06 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315736+0000", 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118052849 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.315880+0000", 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014487300000000001 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316239+0000", 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00035899100000000002 2024-09-16T11:15:00.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50309 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713367050000002, 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11926919, 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.946 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50309 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197685+0000", 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197687+0000", 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4789999999999999e-06 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197690+0000", 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2949999999999998e-06 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197693+0000", 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5689999999999999e-06 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316330+0000", 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118636743 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316511+0000", 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018160999999999999 2024-09-16T11:15:00.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.316954+0000", 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044318500000000002 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50310 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713357169000005, 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119981561, 2024-09-16T11:15:00.950 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50310 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197695+0000", 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197697+0000", 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3400000000000001e-06 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197700+0000", 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3239999999999999e-06 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197702+0000", 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2180000000000002e-06 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317060+0000", 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.119357675 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317234+0000", 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017381800000000001 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317677+0000", 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00044290099999999999 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.954 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713347992999999, 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:00.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:00.958 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713331925999995, 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.959 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:00.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713315531000006, 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:00.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:00.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713305957000003, 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.713296420999995, 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:00.972 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:00.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50320 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.709239764000003, 2024-09-16T11:15:00.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118555891, 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50320 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201812+0000", 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201813+0000", 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201814+0000", 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3680000000000001e-06 2024-09-16T11:15:00.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201817+0000", 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8729999999999999e-06 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201819+0000", 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5380000000000001e-06 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320338+0000", 2024-09-16T11:15:00.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11851816900000001 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320368+0000", 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0686999999999999e-05 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50321 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.709230382000001, 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11863278400000001, 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50321 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201822+0000", 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6100000000000002e-07 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201823+0000", 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.302e-06 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201826+0000", 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8720000000000002e-06 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201828+0000", 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0370000000000001e-06 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320414+0000", 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118585723 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320455+0000", 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0589000000000001e-05 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.984 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50322 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.709221724000002, 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118738332, 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50322 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201830+0000", 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201831+0000", 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201832+0000", 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201835+0000", 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.444e-06 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201838+0000", 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.255e-06 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320546+0000", 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118708724 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320569+0000", 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2504e-05 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50323 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.709212489999999, 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.1187933, 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50323 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201840+0000", 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5100000000000001e-07 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201841+0000", 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201844+0000", 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8559999999999998e-06 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201846+0000", 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.142e-06 2024-09-16T11:15:00.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320622+0000", 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118775743 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.320633+0000", 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1100999999999999e-05 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.015459538000002, 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:00.996 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.329624284000001, 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:00.997 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:00.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:01.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:01.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.003 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:01.005 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:01.005 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:01.001+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:01.106 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:01.101+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:01.107 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:01.107 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:01.107 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:01.108 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:01.117 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:01.206 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:01.201+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:01.259 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:01.259 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:01.259 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:01.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:01.307 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:01.303+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:01.349 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:01.368 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:01.368 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:01.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.588225577, 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:01.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:01.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:01.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:01.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.20029987699999, 2024-09-16T11:15:01.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:01.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:01.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:01.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.200148547, 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:01.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:01.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:01.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:01.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:01.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:01.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:01.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:01.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.20009706100001, 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.394 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:01.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:01.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:01.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:01.400 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.20002691299999, 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.401 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:01.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:01.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:01.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:01.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:01.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:01.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:01.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.411 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.19985444299999, 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:01.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:01.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.19979612099999, 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:01.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:01.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:01.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:01.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.427 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.199644001, 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:01.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:01.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:01.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:01.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.19946548199999, 2024-09-16T11:15:01.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:01.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:01.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:01.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:01.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.445 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.19943250899999, 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:01.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 130.64724478400001, 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:01.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:01.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.457 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 130.64711598700001, 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:01.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:01.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 130.64705764199999, 2024-09-16T11:15:01.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:01.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:01.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 130.64700325300001, 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:01.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:01.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:01.474 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 130.64686894600001, 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.475 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:01.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:01.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 129.86571357700001, 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:15:01.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.486 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 129.865626265, 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.487 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 129.865388137, 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:01.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:01.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 59.185008128, 2024-09-16T11:15:01.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:15:01.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 44.476815692000002, 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:01.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.507 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:01.507 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:01.507 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:01.508 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:01.508 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:01.509 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:01.503+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:01.518 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:01.547 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:01.580 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:01.611 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:01.607+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:01.616 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:01.625 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:01.626 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:01.635 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:01.639 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:01.662 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:01.688 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:15:01.692 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:01.694 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:01.694 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:01.694 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:01.694 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:01.694 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:01.695 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:01.695 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:01.695 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:01.747 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:01.747 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:01.748 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:01.743+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:01.756 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:01.813 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:01.811+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:01.899 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:01.899 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:01.908 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:01.914 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:01.909+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:02.009 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.010 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.015 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:02.011+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:02.019 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:02.116 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:02.109+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:02.122 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.122 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:02.218 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:02.217+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:02.230 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.231 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.240 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:02.319 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:02.315+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:02.339 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.339 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.348 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:02.420 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:02.413+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:02.447 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.447 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.457 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:02.521 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:02.515+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:02.561 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.561 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.571 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:02.622 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:02.619+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:02.670 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.670 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:02.725 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:02.723+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:02.785 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.785 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.795 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:02.823 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:02.819+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:02.856 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone38 from 0xc738f0e (overlap 0x9fa39e) to 0xff6c3 2024-09-16T11:15:02.924 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:02.919+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:02.928 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:02.929 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:02.945 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:03.025 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:03.023+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:03.055 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.055 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.065 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:03.128 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:03.123+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:03.182 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.182 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.192 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:03.227 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:03.223+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:03.265 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-16T11:15:03.289 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.289 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.297 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:03.342 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:03.337+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:03.407 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.407 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:03.429 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:03.427+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:03.519 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.519 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.529 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:03.531 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:03.526+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:03.632 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:03.626+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:03.634 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.634 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.644 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:03.703 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:15:03.704 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:03.733 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:03.725+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:03.757 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.757 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.767 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:03.834 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:03.830+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:03.877 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.877 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:03.887 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:03.935 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:03.930+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:03.999 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:03.999 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:04.009 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:04.035 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:04.030+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:04.072 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:04.096 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:04.109 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:04.109 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:04.119 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:04.136 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:04.136 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:04.130+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:04.164 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:04.196 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:04.222 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:04.222 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:04.223 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:04.233 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:04.253 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:04.277 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:04.273+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:04.283 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:04.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:04.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:04.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:04.288 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:04.288 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:04.288 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:04.288 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:04.288 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:04.338 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:04.333+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:04.339 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.560966842, 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:04.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:04.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17304114199999, 2024-09-16T11:15:04.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:04.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:04.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:04.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17288981199999, 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:04.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:04.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.172838326, 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.359 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:04.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:04.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:04.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17276817800001, 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:04.365 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17259570799999, 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:04.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17253738599999, 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:04.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:04.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17238526599999, 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.383 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:04.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:04.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:04.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17220674699999, 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:04.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.394 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.17217377399999, 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.395 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:04.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 133.619986049, 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:04.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:04.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 133.619857252, 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.407 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:04.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.412 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 133.61979890699999, 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:04.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:04.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 133.619744518, 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.419 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:04.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:04.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.424 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 133.61961021100001, 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.425 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:04.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:04.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 132.838454842, 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.431 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:15:04.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:15:04.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 132.83836753, 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.437 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:04.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:04.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 132.83812940199999, 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:04.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:04.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:15:04.449 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:04.450 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 62.157749393000003, 2024-09-16T11:15:04.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:15:04.450 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.450 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.450 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:15:04.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.449556956999999, 2024-09-16T11:15:04.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:04.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:04.460 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:04.460 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:04.461 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:04.453+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:04.461 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:04.547 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:04.547 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:04.548 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:04.548 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:04.555 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:04.562 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:04.557+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:04.645 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:04.645 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:04.645 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:04.646 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:04.646 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:04.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:04.701 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:04.694+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:04.704 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:15:04.731 INFO:teuthology.orchestra.run.smithi162.stdout:4893 punch from 0x5f168db to 0x5f260c1, (0xf7e6 bytes) 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.576905672999999, 2024-09-16T11:15:04.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:04.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.574442399000006, 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:15:04.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:15:04.764 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.573234163999999, 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.765 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:04.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.769 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.571168872000001, 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:04.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:04.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.571155016000006, 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:04.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:04.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.571141140999998, 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:04.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.571127374, 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:04.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.787 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.571079865000002, 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:04.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:04.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.570794047000007, 2024-09-16T11:15:04.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:04.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:04.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:04.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:04.797 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.478130230999994, 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:04.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:04.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:04.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:04.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.477981048999993, 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:04.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:04.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:04.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:04.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:04.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:04.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:04.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:04.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.48957982900000002, 2024-09-16T11:15:04.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:04.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.48953286899999998, 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:04.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:04.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.48951417699999999, 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:04.820 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:04.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.489497978, 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:04.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:04.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.48948293599999998, 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.830 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:04.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.489467186, 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:04.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.489451896, 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:04.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:04.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:04.842 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.48943631799999998, 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:04.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.48942133900000001, 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:04.847 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:04.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.851 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.852 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:04.852 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:04.852 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:04.854 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:04.854 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:04.850+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:04.954 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:04.949+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:04.958 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:04.958 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:04.958 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:04.958 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:04.968 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:05.055 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:05.050+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:05.071 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:05.071 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:05.071 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:05.071 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:05.071 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:05.080 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:05.156 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:05.149+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:05.186 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.688348183999999, 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:05.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.434935385999999, 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:05.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:05.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:05.198 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.434666069999999, 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.199 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:05.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:05.203 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.434267560000002, 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:05.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.433978772000003, 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.432279356000002, 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:05.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.217 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.432201040000002, 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:05.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:05.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.432167301, 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:05.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.226 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.432126556, 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.227 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:05.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.428926107999999, 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:05.231 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:05.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:05.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.428908051000001, 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:05.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:05.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:05.240 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.428882062, 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:05.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:05.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.428183631000003, 2024-09-16T11:15:05.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:05.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.428171071999998, 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:05.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.518495318999999, 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:05.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:05.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:05.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.83606981400000002, 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:05.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:05.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.83320492099999999, 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:05.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:05.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.82415900499999994, 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:05.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:05.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.76674350000000002, 2024-09-16T11:15:05.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:05.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:05.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.289 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.72048433499999998, 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.290 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.297 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.297 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:05.298 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:05.299 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:05.293+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:05.399 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:05.393+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:05.413 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:05.413 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:05.413 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:05.413 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:05.422 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:05.449 INFO:teuthology.orchestra.run.smithi162.stdout:4894 read 0x4342edd thru 0x434f353 (0xc477 bytes) 2024-09-16T11:15:05.449 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:05.463 INFO:teuthology.orchestra.run.smithi162.stdout:4897 writesame 0xeb609c9 thru 0xeb652a8 data_size 0x16(0x48e0 bytes) 2024-09-16T11:15:05.463 INFO:teuthology.orchestra.run.smithi162.stdout:4898 writesame 0xd2dcfff thru 0xd2ea5f6 data_size 0x4(0xd5f8 bytes) 2024-09-16T11:15:05.476 INFO:teuthology.orchestra.run.smithi162.stdout:4899 punch from 0x4190461 to 0x4194225, (0x3dc4 bytes) 2024-09-16T11:15:05.476 INFO:teuthology.orchestra.run.smithi162.stdout:4901 writesame 0x7117ac1 thru 0x711816e data_size 0x13(0x6ae bytes) 2024-09-16T11:15:05.500 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:05.493+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:05.504 INFO:teuthology.orchestra.run.smithi162.stdout:4902 trunc from 0xeb652a9 to 0x6bf9bd 2024-09-16T11:15:05.535 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:05.535 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:05.535 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:05.535 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:05.535 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:05.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:05.601 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:05.598+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:05.639 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:05.639 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:05.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557774473999999, 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.640 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:05.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:05.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557760016000003, 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.645 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:05.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:05.649 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557742816000001, 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:05.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:05.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:05.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557729295999998, 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:05.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:05.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557720754000002, 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:05.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:05.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557710610000001, 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:05.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:05.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557697103000002, 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:05.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557683511, 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557659229000002, 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.681 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:05.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:05.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557625068, 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.686 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557591670999997, 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:05.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.696 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557569026000003, 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557553450999997, 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:05.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:05.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:05.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.557536417000001, 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:05.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.553903452999997, 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:05.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.716 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.553869110999997, 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:05.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:05.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:05.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.553853165, 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:05.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:05.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.553837553999998, 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:05.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:05.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.553822466, 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:05.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:05.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.553807663999997, 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:05.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:05.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:05.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:05.742 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:05.742 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 6, 7, 2, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:05.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:05.742 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:05.742 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-16T11:15:05.744 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:05.744 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:05.737+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:05.845 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:05.842+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:05.890 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:05.890 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:05.891 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:05.900 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:05.946 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:05.942+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:06.014 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:06.014 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:06.014 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:06.014 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:06.014 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:06.023 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:06.047 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:06.041+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:06.101 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:06.101 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:06.101 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.41593 -> 2.41593 (expected baseline 3.3197) 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 9 -> 9 pgs (0.7 -> 0.7 * mean) 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 17 -> 17 pgs (1.32222 -> 1.32222 * mean) 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 0.05 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:06.102 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0470 2024-09-16T11:15:06.105 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:06.115 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:06.115 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-16T11:15:06.126 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.989442026999996, 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.127 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:06.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:06.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.989423638999995, 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.132 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:06.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:06.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.989122882000004, 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:06.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:06.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.989093177000001, 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:06.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:06.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.989079778999994, 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:06.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:06.149 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.989065975000003, 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:06.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:06.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.988690976000001, 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:06.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.975146945000006, 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:06.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:06.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.975125371000004, 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:06.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:06.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.975108872000007, 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:06.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:06.171 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.975092795999998, 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.974709222000001, 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.176 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:06.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.962627045000005, 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.181 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:06.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:06.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.962618844000005, 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.185 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:06.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:06.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.962610822000002, 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:06.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:06.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 49.279131943000003, 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:06.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.199 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 44.045713591999998, 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:06.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:06.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.659320219000001, 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:06.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 32.707715571000001, 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:06.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:06.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:06.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:06.217 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 25.571918639, 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:06.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:06.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:06.223 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:06.223 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:06.224 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:06.225 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:06.218+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:06.301 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:06.325 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:06.317+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:06.337 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:186019 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.517097+0000", 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81768277700000003, 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.81774075300000004, 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 186019 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517097+0000", 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517097+0000", 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517098+0000", 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8000000000000005e-07 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517102+0000", 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517110+0000", 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4970000000000001e-06 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517116+0000", 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4729999999999999e-06 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517189+0000", 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517197+0000", 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4139999999999997e-06 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:186023 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.517211+0000", 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81756961800000005, 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.81777742200000003, 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 186023 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517211+0000", 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517211+0000", 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517211+0000", 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0400000000000005e-07 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517215+0000", 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.613e-06 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517229+0000", 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4406999999999999e-05 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517236+0000", 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0489999999999998e-06 2024-09-16T11:15:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517310+0000", 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4060000000000006e-05 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517342+0000", 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1180999999999998e-05 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:186024 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.517261+0000", 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81751879900000002, 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.81782474199999999, 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 186024 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517261+0000", 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517261+0000", 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517262+0000", 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517268+0000", 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5559999999999998e-06 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517279+0000", 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0804e-05 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517290+0000", 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0910000000000001e-05 2024-09-16T11:15:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517402+0000", 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111869 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517421+0000", 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9741999999999998e-05 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:186029 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.517424+0000", 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81735614400000001, 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.81775561600000002, 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 186029 2024-09-16T11:15:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517424+0000", 2024-09-16T11:15:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517424+0000", 2024-09-16T11:15:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517425+0000", 2024-09-16T11:15:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.207e-06 2024-09-16T11:15:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517431+0000", 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.553e-06 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517446+0000", 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5262999999999999e-05 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517452+0000", 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517515+0000", 2024-09-16T11:15:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3075000000000002e-05 2024-09-16T11:15:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517526+0000", 2024-09-16T11:15:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.131e-05 2024-09-16T11:15:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:186033 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.517530+0000", 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81725023500000005, 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.81774328200000002, 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 186033 2024-09-16T11:15:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517530+0000", 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517530+0000", 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517531+0000", 2024-09-16T11:15:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.47e-07 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517539+0000", 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4700000000000002e-06 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517548+0000", 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1360000000000003e-06 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517554+0000", 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5169999999999999e-06 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517597+0000", 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3495000000000003e-05 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.517603+0000", 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7760000000000003e-06 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185909 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515087+0000", 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.819692804, 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.82027883700000004, 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185909 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515087+0000", 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515087+0000", 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515088+0000", 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1739999999999999e-06 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515093+0000", 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5310000000000003e-06 2024-09-16T11:15:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515103+0000", 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0332e-05 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515109+0000", 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.122e-06 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515230+0000", 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012087 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515248+0000", 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7387000000000001e-05 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185910 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515132+0000", 2024-09-16T11:15:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81964834499999994, 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.82033511999999997, 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185910 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515132+0000", 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515132+0000", 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515133+0000", 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8299999999999996e-07 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515136+0000", 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2629999999999999e-06 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515147+0000", 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1554e-05 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515157+0000", 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5370000000000003e-06 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515296+0000", 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013957299999999999 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515308+0000", 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1143e-05 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185918 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515314+0000", 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81946616400000005, 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.82034747299999999, 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185918 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515314+0000", 2024-09-16T11:15:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515314+0000", 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515315+0000", 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3699999999999999e-07 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515318+0000", 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2169999999999999e-06 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515326+0000", 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2230000000000007e-06 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515331+0000", 2024-09-16T11:15:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2109999999999997e-06 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515407+0000", 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5945999999999994e-05 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515417+0000", 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7270000000000002e-06 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185952 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515880+0000", 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81889990199999996, 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.81994186499999999, 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185952 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515880+0000", 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515880+0000", 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515882+0000", 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3170000000000001e-06 2024-09-16T11:15:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515886+0000", 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1999999999999996e-06 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515897+0000", 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1634000000000001e-05 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515904+0000", 2024-09-16T11:15:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2310000000000001e-06 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515981+0000", 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7756000000000006e-05 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516000+0000", 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8337000000000001e-05 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185957 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515976+0000", 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81880403499999999, 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.82003616999999995, 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185957 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515976+0000", 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515976+0000", 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515977+0000", 2024-09-16T11:15:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.04e-06 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515982+0000", 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.186e-06 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515993+0000", 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.024e-05 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515998+0000", 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7509999999999997e-06 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516086+0000", 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7540000000000006e-05 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516095+0000", 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2229999999999996e-06 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185962 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516071+0000", 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81870941900000005, 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.82004268199999997, 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185962 2024-09-16T11:15:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516071+0000", 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516071+0000", 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516072+0000", 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1429999999999999e-06 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516079+0000", 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8279999999999997e-06 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516098+0000", 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9074999999999999e-05 2024-09-16T11:15:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516109+0000", 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1262e-05 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516233+0000", 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012363000000000001 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516254+0000", 2024-09-16T11:15:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1009000000000002e-05 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185963 4.9 4.9c6ec529 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516168+0000", 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.81861237200000003, 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.82003817400000001, 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185963 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516168+0000", 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516168+0000", 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516170+0000", 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.666e-06 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2269999999999999e-05 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516210+0000", 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7093999999999998e-05 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516222+0000", 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2539e-05 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516332+0000", 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010959900000000001 2024-09-16T11:15:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516350+0000", 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8340000000000001e-05 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 12 2024-09-16T11:15:06.392 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:06.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:06.412 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:06.412 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.41593 -> 2.41593 (expected baseline 3.3197) 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 9 -> 9 pgs (0.7 -> 0.7 * mean) 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 17 -> 17 pgs (1.32222 -> 1.32222 * mean) 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:06.413 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:06.414 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0470 2024-09-16T11:15:06.416 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:06.425 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:06.421+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:06.426 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:15:06.427 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-16T11:15:06.428 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:06.453 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:06.476 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:06.493 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:06.507 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:06.507 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:06.507 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:06.507 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:06.507 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:06.513 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:06.517 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:06.526 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:06.522+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:06.534 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:06.561 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:06.591 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:06.594 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:06.645 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:06.642+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:06.662 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:06.662 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:06.672 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:06.716 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:15:06.719 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:15:06.714+0000 7f228efd7700 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:moved 4 / 90 (4.44444%) 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.41593 -> 1.45686 (expected baseline 3.3197) 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 9 -> 10 pgs (0.7 -> 0.777778 * mean) 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 17 -> 13 pgs (1.32222 -> 1.01111 * mean) 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:06.719 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 3 2024-09-16T11:15:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0052 2024-09-16T11:15:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 weight 1.0000 -> 0.8099 2024-09-16T11:15:06.720 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 weight 1.0000 -> 0.8945 2024-09-16T11:15:06.722 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:06.728 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:06.721+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:06.736 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:06.736 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T11:15:06.771 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:06.771 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:06.781 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:06.829 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:06.822+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:06.890 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:06.890 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:06.900 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:06.930 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:06.925+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:07.003 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.003 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.017 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:07.031 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:07.026+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:07.124 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.124 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.131 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:07.126+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:07.133 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:07.232 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.232 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.233 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:07.226+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:07.243 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:07.333 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:07.329+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:07.346 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.346 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.356 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:07.435 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:07.429+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:07.445 INFO:teuthology.orchestra.run.smithi162.stdout:moved 2 / 90 (2.22222%) 2024-09-16T11:15:07.445 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:07.445 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.41593 -> 1.72615 (expected baseline 3.3197) 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 9 -> 10 pgs (0.7 -> 0.777778 * mean) 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 17 -> 15 pgs (1.32222 -> 1.16667 * mean) 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 0.05 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0052 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-16T11:15:07.446 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-16T11:15:07.454 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:15:07.464 INFO:teuthology.orchestra.run.smithi162.stdout:4907 punch from 0x5b5e79 to 0x5b70ed, (0x1274 bytes) 2024-09-16T11:15:07.464 INFO:teuthology.orchestra.run.smithi162.stdout:4908 read 0x41ef73 thru 0x426bef (0x7c7d bytes) 2024-09-16T11:15:07.465 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.465 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.470 INFO:teuthology.orchestra.run.smithi162.stdout:4909 trunc from 0x6bf9bd to 0x21e6580 2024-09-16T11:15:07.476 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:07.477 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:07.478 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-16T11:15:07.507 INFO:teuthology.orchestra.run.smithi162.stdout:4910 writesame 0x6638146 thru 0x6645d24 data_size 0x2b(0xdbdf bytes) 2024-09-16T11:15:07.507 INFO:teuthology.orchestra.run.smithi162.stdout:4911 write 0x56ba21a thru 0x56c75c6 (0xd3ad bytes) 2024-09-16T11:15:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:4914 punch from 0x62c084 to 0x62f6e2, (0x365e bytes) 2024-09-16T11:15:07.516 INFO:teuthology.orchestra.run.smithi162.stdout:4915 read 0x275cd53 thru 0x2768d0e (0xbfbc bytes) 2024-09-16T11:15:07.533 INFO:teuthology.orchestra.run.smithi162.stdout:4916 write 0x8e9c1f6 thru 0x8ea1976 (0x5781 bytes) 2024-09-16T11:15:07.535 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:07.530+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:07.543 INFO:teuthology.orchestra.run.smithi162.stdout:4917 read 0x14d6e1f thru 0x14de76b (0x794d bytes) 2024-09-16T11:15:07.544 INFO:teuthology.orchestra.run.smithi162.stdout:4921 writesame 0x5bd4361 thru 0x5bd677f data_size 0x1(0x241f bytes) 2024-09-16T11:15:07.562 INFO:teuthology.orchestra.run.smithi162.stdout:4922 trunc from 0x8ea1977 to 0x5a187c8 2024-09-16T11:15:07.588 INFO:teuthology.orchestra.run.smithi162.stdout:4924 trunc from 0x5a187c8 to 0xb798add 2024-09-16T11:15:07.600 INFO:teuthology.orchestra.run.smithi162.stdout:4925 trunc from 0xb798add to 0xbdbf872 2024-09-16T11:15:07.611 INFO:teuthology.orchestra.run.smithi162.stdout:4926 punch from 0xb50e000 to 0xb5119b7, (0x39b7 bytes) 2024-09-16T11:15:07.611 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:07.611 INFO:teuthology.orchestra.run.smithi162.stdout:4928 punch from 0x21502de to 0x2156036, (0x5d58 bytes) 2024-09-16T11:15:07.611 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:07.611 INFO:teuthology.orchestra.run.smithi162.stdout:4930 write 0x120973a thru 0x1210717 (0x6fde bytes) 2024-09-16T11:15:07.619 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.619 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.628 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:07.636 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:07.629+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:07.722 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.722 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.731 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:07.777 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:07.774+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:07.792 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:07.792 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:07.792 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 1.72615 -> 1.72615 (expected baseline 3.3197) 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 15 -> 15 pgs (1.16667 -> 1.16667 * mean) 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 0.05 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:07.793 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0470 2024-09-16T11:15:07.796 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:07.835 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.836 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.838 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:07.834+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:07.845 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:07.939 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:07.937+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:07.944 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:07.944 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:07.953 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:08.040 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:08.037+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:08.055 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.055 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.064 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:08.141 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:08.138+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:08.161 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.161 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.170 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:08.242 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:08.238+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:08.318 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.319 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.328 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:08.343 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:08.337+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:08.428 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.429 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.438 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:08.444 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:08.437+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:08.544 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:08.538+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:08.547 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.547 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.557 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:08.605 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:08.650 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:08.646+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:08.650 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.651 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.662 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:08.739 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:08.746 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:08.742+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:08.759 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:08.759 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:08.760 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:08.768 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:08.783 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:08.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:08.828 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:08.847 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:08.841+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:08.855 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:08.875 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:08.879 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:08.879 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:08.880 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:08.880 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:08.889 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:08.897 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:08.899 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:08.900 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:08.953 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:08.950+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:08.992 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:08.993 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:08.993 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:08.993 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:08.993 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:09.001 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:09.049 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:09.046+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:09.102 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.964566422000004, 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:09.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.964548034000003, 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:09.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:09.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.964247276999998, 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:09.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.964217571999995, 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:09.115 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:09.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.964204174000002, 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.120 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:09.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.964190369999997, 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:09.124 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.128 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.963815370999995, 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:09.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:09.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.131 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.132 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.95027134, 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:09.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.950249765999999, 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.137 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:09.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:09.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.950233267000002, 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.142 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:09.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.146 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.950217190999993, 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.949833616999996, 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.151 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.155 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.93775144, 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:09.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.159 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.937743239, 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:09.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:09.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.937735216999997, 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:09.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:09.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.254256337999998, 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:09.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:09.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:09.174 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.020837987, 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:09.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.634444614000003, 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:09.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 35.682839966000003, 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:09.185 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:09.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.547043034000001, 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:09.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:09.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:09.196 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:09.198 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:09.198 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:09.194+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:09.299 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:09.293+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:09.305 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:09.305 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:09.305 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:09.305 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:09.314 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:09.401 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:09.394+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:09.416 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:09.416 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:09.416 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:09.416 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:09.416 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:09.425 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:09.473 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:15:09.502 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:09.497+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:09.536 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102667 4.d 4:b39dc362:::rbd_data.11b578dab209.0000000000000009:1b [read 1296384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.458012042, 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.079928343, 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.537 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102667 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073991+0000", 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.77e-07 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.073994+0000", 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2330000000000001e-06 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074000+0000", 2024-09-16T11:15:09.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1410000000000002e-06 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074005+0000", 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.293e-06 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083131+0000", 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091257869999999998 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083134+0000", 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3850000000000002e-06 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153682+0000", 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0705472700000001 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153766+0000", 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4802000000000006e-05 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153919+0000", 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000153055 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457992676000003, 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.542 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:09.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.547 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457951913999999, 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:09.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:09.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457921116000001, 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:09.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.558 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.559 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.559 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:09.560 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457902527999998, 2024-09-16T11:15:09.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:09.560 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.560 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:09.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:09.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457884475999997, 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:09.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:09.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457864549, 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.571 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:09.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:09.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:09.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457845710000001, 2024-09-16T11:15:09.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:09.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.581 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457653761000003, 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:09.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:09.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.457634480999999, 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:09.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:09.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102706 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1263616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454164216000002, 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0800743989999999, 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.592 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102706 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077839+0000", 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077841+0000", 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6050000000000001e-06 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077845+0000", 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6920000000000001e-06 2024-09-16T11:15:09.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077848+0000", 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.21e-06 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083543+0000", 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056957129999999998 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083547+0000", 2024-09-16T11:15:09.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157506+0000", 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0739593679999999 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157684+0000", 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017741999999999999 2024-09-16T11:15:09.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157913+0000", 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022970099999999999 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:09.597 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454142912999998, 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:09.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454132842, 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:09.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:09.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454104538999999, 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454090610000002, 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.613 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:09.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454081846999998, 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:09.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:09.623 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454058027999999, 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:09.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:09.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.454026906000003, 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:09.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:09.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.453979167, 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.634 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:09.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:09.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.453957058999997, 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:09.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:09.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:09.643 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:09.645 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:09.645 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:09.638+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:09.746 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:09.741+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:09.809 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:09.809 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:09.809 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:09.809 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:09.819 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:09.846 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:09.842+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:09.924 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:09.924 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:09.924 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:09.925 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:09.925 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:09.934 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:09.947 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:09.941+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:09.988 INFO:teuthology.orchestra.run.smithi162.stdout:4931 trunc from 0xbdbf872 to 0xbbdf109 2024-09-16T11:15:09.998 INFO:teuthology.orchestra.run.smithi162.stdout:4933 punch from 0x2dc8eb6 to 0x2dd2613, (0x975d bytes) 2024-09-16T11:15:10.020 INFO:teuthology.orchestra.run.smithi162.stdout:4934 trunc from 0xbbdf109 to 0x900bee 2024-09-16T11:15:10.040 INFO:teuthology.orchestra.run.smithi162.stdout:4936 punch from 0x541a31 to 0x54ed61, (0xd330 bytes) 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:10.044 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.17943183, 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:10.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:10.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011460181000004, 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.049 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:10.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.053 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011448619999996, 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:10.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011435418000005, 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.058 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:10.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011422977999999, 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.063 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:10.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.067 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011409466999993, 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.068 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:10.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011398552000003, 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:10.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:10.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011372633999997, 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.077 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:10.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011357180000005, 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:10.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:10.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011347294000004, 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.086 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:10.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011337530999995, 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.091 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:10.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011327354000002, 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.100 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011310871999996, 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:10.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011296174999998, 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.105 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:10.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011277731999996, 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:10.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:10.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011260782999997, 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.114 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011241569999996, 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.118 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.122 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011224354000007, 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:10.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:10.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:10.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.127 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011203100000003, 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.131 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.011183131999999, 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:10.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.136 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:10.138 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4940 write 0x71b6cc1 thru 0x71b89b0 (0x1cf0 bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4941 write 0xa93a4dd thru 0xa9469bb (0xc4df bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4944 writesame 0xdcffaa1 thru 0xdd0ca46 data_size 0x7(0xcfa6 bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4945 punch from 0xd6ed7ba to 0xd6f14dc, (0x3d22 bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4946 punch from 0x3d3d8a6 to 0x3d46aa9, (0x9203 bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4947 write 0x1d7dd22 thru 0x1d87d96 (0xa075 bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4948 writesame 0xa9eea72 thru 0xa9f21e0 data_size 0x1(0x376f bytes) 2024-09-16T11:15:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:4949 write 0x6e77223 thru 0x6e79586 (0x2364 bytes) 2024-09-16T11:15:10.140 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:10.134+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:10.143 INFO:teuthology.orchestra.run.smithi162.stdout:4951 read 0x4d9d548 thru 0x4da271c (0x51d5 bytes) 2024-09-16T11:15:10.143 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:10.156 INFO:teuthology.orchestra.run.smithi162.stdout:4955 trunc from 0xdd0ca47 to 0x6a32e99 2024-09-16T11:15:10.189 INFO:teuthology.orchestra.run.smithi162.stdout:4956 trunc from 0x6a32e99 to 0xbf63ca3 2024-09-16T11:15:10.210 INFO:teuthology.orchestra.run.smithi162.stdout:4957 trunc from 0xbf63ca3 to 0x8c353eb 2024-09-16T11:15:10.234 INFO:teuthology.orchestra.run.smithi162.stdout:4958 writesame 0x93c8949 thru 0x93cf23f data_size 0xd(0x68f7 bytes) 2024-09-16T11:15:10.234 INFO:teuthology.orchestra.run.smithi162.stdout:4959 trunc from 0x93cf240 to 0x90d592f 2024-09-16T11:15:10.239 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:10.234+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:10.245 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:10.261 INFO:teuthology.orchestra.run.smithi162.stdout:4961 writesame 0xa3f67d9 thru 0xa3f91d0 data_size 0x11(0x29f8 bytes) 2024-09-16T11:15:10.261 INFO:teuthology.orchestra.run.smithi162.stdout:4962 read 0x4256183 thru 0x425c189 (0x6007 bytes) 2024-09-16T11:15:10.262 INFO:teuthology.orchestra.run.smithi162.stdout:4963 read 0x1278284 thru 0x127d9dc (0x5759 bytes) 2024-09-16T11:15:10.262 INFO:teuthology.orchestra.run.smithi162.stdout:4964 clone 40 order 19 su 65536 sc 9 2024-09-16T11:15:10.298 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:10.299 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:10.299 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:10.299 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:10.309 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:10.339 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:10.333+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:10.412 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:10.412 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:10.412 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:10.412 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:10.412 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:10.422 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:10.442 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:15:10.466 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:10.461+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:10.541 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:10.538+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.572 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.375587173, 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:10.577 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.371717541999999, 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:10.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:10.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:10.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.371701474999995, 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.582 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:10.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.586 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.371685080000006, 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.587 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:10.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.371675506000003, 2024-09-16T11:15:10.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:10.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:10.596 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 82.371665969999995, 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.597 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:10.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 53.673829087000001, 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.987993832999997, 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.605 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:10.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:10.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0534744900000002, 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.612 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:10.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:10.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:10.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0533938190000001, 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:10.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:10.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0532724269999996, 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:10.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:10.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:10.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0532202330000002, 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:10.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:10.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0531635000000001, 2024-09-16T11:15:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.638 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0531163010000002, 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:10.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:10.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0530663310000001, 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.644 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:10.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:10.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:10.649 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.052999217, 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:10.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.654 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0510241310000001, 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:10.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.659 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0509756460000004, 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:10.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:10.664 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0508586180000004, 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.665 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 5.0507674429999998, 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:10.670 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:10.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:10.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:10.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:10.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:10.675 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:10.676 INFO:teuthology.orchestra.run.smithi162.stdout:truncating image image_client.0-clone39 from 0xa3f91d1 (overlap 0x6bf9bd) to 0x566164 2024-09-16T11:15:10.677 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:10.677 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:10.669+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:10.778 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:10.773+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:10.786 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:10.786 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:10.795 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:10.878 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:10.874+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:10.891 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:10.891 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:10.891 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:10.891 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:10.891 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:10.901 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:10.911 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:10.980 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:10.973+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:10.999 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 141.21986046000001, 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.000 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:11.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:11.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.83193476, 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.008 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:11.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.83178343, 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.016 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.016 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:11.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.016 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:11.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:11.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.83173194400001, 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:11.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:11.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.027 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.83166179599999, 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:11.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.831489326, 2024-09-16T11:15:11.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:11.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:11.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:11.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.831431004, 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.040 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:11.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:11.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.046 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.831278884, 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:11.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:11.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.831100365, 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:11.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:11.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.83106739199999, 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:11.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:11.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:11.064 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.27887966700001, 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.065 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:11.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:11.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.071 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.27875087000001, 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:11.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.278692525, 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:11.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:11.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:11.083 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.27863813600001, 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:11.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:11.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.089 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.27850382899999, 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:11.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:11.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:11.096 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 139.49734846000001, 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:11.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:15:11.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 139.49726114800001, 2024-09-16T11:15:11.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:11.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 139.49702302, 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:11.108 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:11.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:11.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.115 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:15:11.116 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 68.816643010999996, 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:11.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:15:11.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 54.108450574999999, 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:11.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:11.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:11.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.125 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:11.126 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:11.126 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:11.127 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:11.127 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:11.127 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:11.127 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:11.128 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:11.121+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:11.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:11.139 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:11.164 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:11.186 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:11.216 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:11.219 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:11.228 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:11.221+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:11.234 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.235 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:11.329 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:11.322+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:11.341 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.341 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.350 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:11.429 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:11.422+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:11.460 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.460 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:11.530 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:11.525+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:11.568 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.569 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.578 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:11.631 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:11.626+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.674 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:11.731 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:11.726+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:11.785 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.786 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.795 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:11.842 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:11.838+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:11.892 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:11.893 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:11.901 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:11.932 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:11.926+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:12.001 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.001 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.011 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:12.032 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:12.026+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:12.113 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.114 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.123 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:12.165 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:12.162+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:12.234 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:12.229+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:12.263 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.263 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.272 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:12.335 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:12.330+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:12.370 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.370 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.379 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:12.436 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:12.429+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:12.458 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:15:12.493 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-16T11:15:12.517 INFO:teuthology.orchestra.run.smithi162.stdout:4965 trunc from 0xa3f91d1 to 0x9076bb7 2024-09-16T11:15:12.528 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.528 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.537 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:12.534+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:12.541 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:12.556 INFO:teuthology.orchestra.run.smithi162.stdout:4966 trunc from 0x9076bb7 to 0x81424cf 2024-09-16T11:15:12.587 INFO:teuthology.orchestra.run.smithi162.stdout:4967 read 0xd596aa thru 0xd68312 (0xec69 bytes) 2024-09-16T11:15:12.587 INFO:teuthology.orchestra.run.smithi162.stdout:4968 read 0x4e0fd51 thru 0x4e1e0b1 (0xe361 bytes) 2024-09-16T11:15:12.587 INFO:teuthology.orchestra.run.smithi162.stdout:4969 writesame 0x2d7d3a1 thru 0x2d822aa data_size 0x2(0x4f0a bytes) 2024-09-16T11:15:12.604 INFO:teuthology.orchestra.run.smithi162.stdout:4970 punch from 0x6e06728 to 0x6e09f20, (0x37f8 bytes) 2024-09-16T11:15:12.604 INFO:teuthology.orchestra.run.smithi162.stdout:4971 write 0x7902385 thru 0x7906394 (0x4010 bytes) 2024-09-16T11:15:12.610 INFO:teuthology.orchestra.run.smithi162.stdout:4972 writesame 0x3c2b826 thru 0x3c38783 data_size 0x26(0xcf5e bytes) 2024-09-16T11:15:12.618 INFO:teuthology.orchestra.run.smithi162.stdout:4973 write 0x24715c9 thru 0x247daee (0xc526 bytes) 2024-09-16T11:15:12.638 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:12.634+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:12.640 INFO:teuthology.orchestra.run.smithi162.stdout:4975 write 0xa80698e thru 0xa810522 (0x9b95 bytes) 2024-09-16T11:15:12.648 INFO:teuthology.orchestra.run.smithi162.stdout:4976 write 0x6b125f3 thru 0x6b1f1ab (0xcbb9 bytes) 2024-09-16T11:15:12.650 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.650 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.659 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:12.670 INFO:teuthology.orchestra.run.smithi162.stdout:4977 writesame 0xc897045 thru 0xc89d927 data_size 0x1(0x68e3 bytes) 2024-09-16T11:15:12.682 INFO:teuthology.orchestra.run.smithi162.stdout:4979 writesame 0xd8cc04b thru 0xd8d97cb data_size 0x1(0xd781 bytes) 2024-09-16T11:15:12.682 INFO:teuthology.orchestra.run.smithi162.stdout:4980 write 0xa07b0d thru 0xa0ddb6 (0x62aa bytes) 2024-09-16T11:15:12.685 INFO:teuthology.orchestra.run.smithi162.stdout:4981 punch from 0x2b7f14 to 0x2c5974, (0xda60 bytes) 2024-09-16T11:15:12.704 INFO:teuthology.orchestra.run.smithi162.stdout:4982 trunc from 0xd8d97cc to 0x1b6c9e0 2024-09-16T11:15:12.739 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:12.734+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:12.766 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.766 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.769 INFO:teuthology.orchestra.run.smithi162.stdout:4983 write 0xd8fbcac thru 0xd905b02 (0x9e57 bytes) 2024-09-16T11:15:12.776 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:12.776 INFO:teuthology.orchestra.run.smithi162.stdout:4986 write 0x20a00d1 thru 0x20a3aba (0x39ea bytes) 2024-09-16T11:15:12.781 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:12.782 INFO:teuthology.orchestra.run.smithi162.stdout:4987 trunc from 0xd905b03 to 0xe90ceeb 2024-09-16T11:15:12.791 INFO:teuthology.orchestra.run.smithi162.stdout:4988 write 0xbd46ecb thru 0xbd53021 (0xc157 bytes) 2024-09-16T11:15:12.799 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:12.799 INFO:teuthology.orchestra.run.smithi162.stdout:4991 punch from 0x41b8925 to 0x41c2b9d, (0xa278 bytes) 2024-09-16T11:15:12.799 INFO:teuthology.orchestra.run.smithi162.stdout:4992 read 0x94ed5a7 thru 0x94eedda (0x1834 bytes) 2024-09-16T11:15:12.799 INFO:teuthology.orchestra.run.smithi162.stdout:4994 writesame 0xdbdb2cd thru 0xdbea74a data_size 0x16(0xf47e bytes) 2024-09-16T11:15:12.807 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 6, 7, 2, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:12.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:12.807 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:15:12.807 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-16T11:15:12.831 INFO:teuthology.orchestra.run.smithi162.stdout:4995 trunc from 0xe90ceeb to 0xd6513e 2024-09-16T11:15:12.841 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:12.838+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:12.883 INFO:teuthology.orchestra.run.smithi162.stdout:4996 writesame 0x8358a36 thru 0x835e9cd data_size 0x1c(0x5f98 bytes) 2024-09-16T11:15:12.884 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:12.884 INFO:teuthology.orchestra.run.smithi162.stdout:4998 write 0x5bba94f thru 0x5bc2e50 (0x8502 bytes) 2024-09-16T11:15:12.884 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.884 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.893 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:12.912 INFO:teuthology.orchestra.run.smithi162.stdout:4999 write 0xd888277 thru 0xd8972e4 (0xf06e bytes) 2024-09-16T11:15:12.919 INFO:teuthology.orchestra.run.smithi162.stdout:5000 read 0x1c3125b thru 0x1c3ff7c (0xed22 bytes) 2024-09-16T11:15:12.919 INFO:teuthology.orchestra.run.smithi162.stdout:5002 read 0x943c57 thru 0x949793 (0x5b3d bytes) 2024-09-16T11:15:12.919 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:12.919 INFO:teuthology.orchestra.run.smithi162.stdout:5004 write 0x4350712 thru 0x4352e9b (0x278a bytes) 2024-09-16T11:15:12.925 INFO:teuthology.orchestra.run.smithi162.stdout:5005 punch from 0x5f98788 to 0x5f99dc5, (0x163d bytes) 2024-09-16T11:15:12.926 INFO:teuthology.orchestra.run.smithi162.stdout:5006 write 0x87da7d5 thru 0x87e678e (0xbfba bytes) 2024-09-16T11:15:12.932 INFO:teuthology.orchestra.run.smithi162.stdout:5008 write 0x2b65e9 thru 0x2b8745 (0x215d bytes) 2024-09-16T11:15:12.942 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:12.937+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:12.944 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:12.944 INFO:teuthology.orchestra.run.smithi162.stdout:5010 write 0x29e46d4 thru 0x29e544a (0xd77 bytes) 2024-09-16T11:15:12.948 INFO:teuthology.orchestra.run.smithi162.stdout:5011 punch from 0x20700ff to 0x207911d, (0x901e bytes) 2024-09-16T11:15:12.948 INFO:teuthology.orchestra.run.smithi162.stdout:5012 read 0x35c5812 thru 0x35c74ab (0x1c9a bytes) 2024-09-16T11:15:12.949 INFO:teuthology.orchestra.run.smithi162.stdout:5013 punch from 0xcec3112 to 0xcec877a, (0x5668 bytes) 2024-09-16T11:15:12.949 INFO:teuthology.orchestra.run.smithi162.stdout:5015 punch from 0xc07a850 to 0xc089633, (0xede3 bytes) 2024-09-16T11:15:12.949 INFO:teuthology.orchestra.run.smithi162.stdout:5016 read 0xb858a65 thru 0xb859a4a (0xfe6 bytes) 2024-09-16T11:15:12.949 INFO:teuthology.orchestra.run.smithi162.stdout:5017 read 0xba2c1b9 thru 0xba37ec0 (0xbd08 bytes) 2024-09-16T11:15:12.949 INFO:teuthology.orchestra.run.smithi162.stdout:5018 write 0x62fa203 thru 0x62fd39a (0x3198 bytes) 2024-09-16T11:15:12.966 INFO:teuthology.orchestra.run.smithi162.stdout:5019 trunc from 0xd8972e5 to 0x5a21111 2024-09-16T11:15:12.988 INFO:teuthology.orchestra.run.smithi162.stdout:5020 read 0x51346c5 thru 0x513e40d (0x9d49 bytes) 2024-09-16T11:15:12.988 INFO:teuthology.orchestra.run.smithi162.stdout:5021 read 0x26393f7 thru 0x263e666 (0x5270 bytes) 2024-09-16T11:15:12.988 INFO:teuthology.orchestra.run.smithi162.stdout:5022 write 0x5635046 thru 0x563baab (0x6a66 bytes) 2024-09-16T11:15:12.993 INFO:teuthology.orchestra.run.smithi162.stdout:5023 write 0x45ca58b thru 0x45cc2aa (0x1d20 bytes) 2024-09-16T11:15:12.993 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:12.994 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:12.999 INFO:teuthology.orchestra.run.smithi162.stdout:5026 punch from 0x7afeb2 to 0x7ba0c6, (0xa214 bytes) 2024-09-16T11:15:12.999 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:12.999 INFO:teuthology.orchestra.run.smithi162.stdout:5028 write 0x271a869 thru 0x27295ad (0xed45 bytes) 2024-09-16T11:15:13.004 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:13.008 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.008 INFO:teuthology.orchestra.run.smithi162.stdout:5030 read 0x1e881f1 thru 0x1e8ff0d (0x7d1d bytes) 2024-09-16T11:15:13.008 INFO:teuthology.orchestra.run.smithi162.stdout:5032 write 0x31320fb thru 0x3137993 (0x5899 bytes) 2024-09-16T11:15:13.013 INFO:teuthology.orchestra.run.smithi162.stdout:5033 write 0x55c2b56 thru 0x55c9ec1 (0x736c bytes) 2024-09-16T11:15:13.029 INFO:teuthology.orchestra.run.smithi162.stdout:5035 trunc from 0x5a21111 to 0xafb5eed 2024-09-16T11:15:13.041 INFO:teuthology.orchestra.run.smithi162.stdout:5036 read 0x800b9c3 thru 0x8016cdd (0xb31b bytes) 2024-09-16T11:15:13.042 INFO:teuthology.orchestra.run.smithi162.stdout:5038 read 0x41b7d31 thru 0x41c5efb (0xe1cb bytes) 2024-09-16T11:15:13.042 INFO:teuthology.orchestra.run.smithi162.stdout:5039 write 0x546ae thru 0x645ae (0xff01 bytes) 2024-09-16T11:15:13.046 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:13.041+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:13.060 INFO:teuthology.orchestra.run.smithi162.stdout:5041 writesame 0x280bdb9 thru 0x2811f5e data_size 0x1d(0x61a6 bytes) 2024-09-16T11:15:13.075 INFO:teuthology.orchestra.run.smithi162.stdout:5042 write 0xbf1543c thru 0xbf21486 (0xc04b bytes) 2024-09-16T11:15:13.081 INFO:teuthology.orchestra.run.smithi162.stdout:5044 read 0x8ad27e2 thru 0x8ad353f (0xd5e bytes) 2024-09-16T11:15:13.081 INFO:teuthology.orchestra.run.smithi162.stdout:5048 read 0x9336c1b thru 0x933cbfa (0x5fe0 bytes) 2024-09-16T11:15:13.081 INFO:teuthology.orchestra.run.smithi162.stdout:5050 write 0x3c2d9f8 thru 0x3c3d98d (0xff96 bytes) 2024-09-16T11:15:13.086 INFO:teuthology.orchestra.run.smithi162.stdout:5052 write 0x724e61f thru 0x7255730 (0x7112 bytes) 2024-09-16T11:15:13.093 INFO:teuthology.orchestra.run.smithi162.stdout:5053 read 0x3dc7f29 thru 0x3dd397d (0xba55 bytes) 2024-09-16T11:15:13.093 INFO:teuthology.orchestra.run.smithi162.stdout:5054 read 0x48765d5 thru 0x4876837 (0x263 bytes) 2024-09-16T11:15:13.093 INFO:teuthology.orchestra.run.smithi162.stdout:5055 writesame 0x8bedd0d thru 0x8bf7cf4 data_size 0x56(0x9fe8 bytes) 2024-09-16T11:15:13.100 INFO:teuthology.orchestra.run.smithi162.stdout:5056 read 0x66ce868 thru 0x66d7134 (0x88cd bytes) 2024-09-16T11:15:13.100 INFO:teuthology.orchestra.run.smithi162.stdout:5057 write 0x5f7c5c0 thru 0x5f7dc4f (0x1690 bytes) 2024-09-16T11:15:13.105 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.105 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.119 INFO:teuthology.orchestra.run.smithi162.stdout:5062 write 0xcaacc64 thru 0xcab0fee (0x438b bytes) 2024-09-16T11:15:13.126 INFO:teuthology.orchestra.run.smithi162.stdout:5064 read 0x17e1f60 thru 0x17e47e3 (0x2884 bytes) 2024-09-16T11:15:13.127 INFO:teuthology.orchestra.run.smithi162.stdout:5065 writesame 0x9480e33 thru 0x9485c91 data_size 0x1(0x4e5f bytes) 2024-09-16T11:15:13.127 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:15:13.122+0000 7f228efd7700 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:moved 2 / 90 (2.22222%) 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 1.72615 -> 1.45686 (expected baseline 3.3197) 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 15 -> 13 pgs (1.16667 -> 1.01111 * mean) 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0048 2024-09-16T11:15:13.128 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0053 2024-09-16T11:15:13.129 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 weight 0.9500 -> 0.7830 2024-09-16T11:15:13.131 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:13.140 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:13.140 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:13.143 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:13.143 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-16T11:15:13.144 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:13.137+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:13.145 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.146 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.146 INFO:teuthology.orchestra.run.smithi162.stdout:5070 flatten 2024-09-16T11:15:13.151 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:13.206 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.206 INFO:teuthology.orchestra.run.smithi162.stdout:5072 punch from 0x732a49e to 0x7331749, (0x72ab bytes) 2024-09-16T11:15:13.206 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.206 INFO:teuthology.orchestra.run.smithi162.stdout:5074 punch from 0x1f3f383 to 0x1f486e0, (0x935d bytes) 2024-09-16T11:15:13.206 INFO:teuthology.orchestra.run.smithi162.stdout:5075 writesame 0x3f1017b thru 0x3f160ac data_size 0x5(0x5f32 bytes) 2024-09-16T11:15:13.212 INFO:teuthology.orchestra.run.smithi162.stdout:5077 punch from 0x36409f6 to 0x364d119, (0xc723 bytes) 2024-09-16T11:15:13.212 INFO:teuthology.orchestra.run.smithi162.stdout:5078 write 0x1cdb53e thru 0x1ce2851 (0x7314 bytes) 2024-09-16T11:15:13.218 INFO:teuthology.orchestra.run.smithi162.stdout:5079 write 0xa859d7 thru 0xa8d7b0 (0x7dda bytes) 2024-09-16T11:15:13.221 INFO:teuthology.orchestra.run.smithi162.stdout:5080 write 0x50a8acb thru 0x50b2411 (0x9947 bytes) 2024-09-16T11:15:13.228 INFO:teuthology.orchestra.run.smithi162.stdout:5081 punch from 0x62971a2 to 0x6299da4, (0x2c02 bytes) 2024-09-16T11:15:13.229 INFO:teuthology.orchestra.run.smithi162.stdout:5082 write 0x7418734 thru 0x741b2d7 (0x2ba4 bytes) 2024-09-16T11:15:13.230 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:13.244 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:13.238+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:13.247 INFO:teuthology.orchestra.run.smithi162.stdout:5083 trunc from 0xcab0fef to 0x5ee7b71 2024-09-16T11:15:13.264 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:13.264 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:13.268 INFO:teuthology.orchestra.run.smithi162.stdout:5084 write 0xeecd2a thru 0xef07b0 (0x3a87 bytes) 2024-09-16T11:15:13.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:13.299 INFO:teuthology.orchestra.run.smithi162.stdout:5085 write 0xeab45c1 thru 0xeabb903 (0x7343 bytes) 2024-09-16T11:15:13.305 INFO:teuthology.orchestra.run.smithi162.stdout:5086 write 0x2e20ba thru 0x2e490c (0x2853 bytes) 2024-09-16T11:15:13.312 INFO:teuthology.orchestra.run.smithi162.stdout:5087 trunc from 0xeabb904 to 0xc172463 2024-09-16T11:15:13.332 INFO:teuthology.orchestra.run.smithi162.stdout:5088 read 0x1c2b331 thru 0x1c3b2a0 (0xff70 bytes) 2024-09-16T11:15:13.332 INFO:teuthology.orchestra.run.smithi162.stdout:5089 write 0x7bbf00f thru 0x7bc6300 (0x72f2 bytes) 2024-09-16T11:15:13.341 INFO:teuthology.orchestra.run.smithi162.stdout:5090 writesame 0x32af7c2 thru 0x32b55c1 data_size 0x4(0x5e00 bytes) 2024-09-16T11:15:13.345 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:13.342+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:13.354 INFO:teuthology.orchestra.run.smithi162.stdout:5092 read 0x15231d5 thru 0x1531a3e (0xe86a bytes) 2024-09-16T11:15:13.489 INFO:teuthology.orchestra.run.smithi162.stdout:5093 read 0x9ae04cc thru 0x9ae7fed (0x7b22 bytes) 2024-09-16T11:15:13.489 INFO:teuthology.orchestra.run.smithi162.stdout:5094 punch from 0xd57524 to 0xd63625, (0xc101 bytes) 2024-09-16T11:15:13.489 INFO:teuthology.orchestra.run.smithi162.stdout:5095 write 0x331a1ac thru 0x332344f (0x92a4 bytes) 2024-09-16T11:15:13.490 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:13.491 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:13.492 INFO:teuthology.orchestra.run.smithi162.stdout:5097 trunc from 0xc172463 to 0x59b84ab 2024-09-16T11:15:13.492 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:13.492 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:13.493 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:13.493 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:13.493 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:13.485+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:13.494 INFO:teuthology.orchestra.run.smithi162.stdout:5098 write 0x1c79cae thru 0x1c85fd6 (0xc329 bytes) 2024-09-16T11:15:13.494 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:13.494 INFO:teuthology.orchestra.run.smithi162.stdout:5099 writesame 0x8fe1051 thru 0x8fe188c data_size 0x1f(0x83c bytes) 2024-09-16T11:15:13.494 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:13.495 INFO:teuthology.orchestra.run.smithi162.stdout:5100 write 0xc7c855d thru 0xc7d6f5f (0xea03 bytes) 2024-09-16T11:15:13.495 INFO:teuthology.orchestra.run.smithi162.stdout:5101 punch from 0x120dd29 to 0x12175e2, (0x98b9 bytes) 2024-09-16T11:15:13.495 INFO:teuthology.orchestra.run.smithi162.stdout:5105 read 0x85146ed thru 0x851e807 (0xa11b bytes) 2024-09-16T11:15:13.495 INFO:teuthology.orchestra.run.smithi162.stdout:5109 writesame 0x1d99b5b thru 0x1da044c data_size 0x26(0x68f2 bytes) 2024-09-16T11:15:13.495 INFO:teuthology.orchestra.run.smithi162.stdout:5110 write 0x155d44 thru 0x1625c9 (0xc886 bytes) 2024-09-16T11:15:13.496 INFO:teuthology.orchestra.run.smithi162.stdout:5112 write 0x8cf7320 thru 0x8cfdd38 (0x6a19 bytes) 2024-09-16T11:15:13.496 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:13.496 INFO:teuthology.orchestra.run.smithi162.stdout:5113 write 0xd840219 thru 0xd84c246 (0xc02e bytes) 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:5114 write 0x4eb60d6 thru 0x4eb6d45 (0xc70 bytes) 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:5115 read 0xd2f66f9 thru 0xd3009cc (0xa2d4 bytes) 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:5117 punch from 0x16c2013 to 0x16c3e9b, (0x1e88 bytes) 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:5118 read 0x35f11eb thru 0x35f21c2 (0xfd8 bytes) 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:moved 2 / 90 (2.22222%) 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:13.497 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 1.72615 -> 1.45686 (expected baseline 3.3197) 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.1 with 15 -> 13 pgs (1.16667 -> 1.01111 * mean) 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 3 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0048 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0053 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1 weight 0.9500 -> 0.7830 2024-09-16T11:15:13.498 INFO:teuthology.orchestra.run.smithi162.stdout:5119 write 0x47182f5 thru 0x4723459 (0xb165 bytes) 2024-09-16T11:15:13.499 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:15:13.499 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:13.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T11:15:13.501 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:13.518 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:13.520 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:13.520 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:13.520 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:13.520 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:13.520 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:13.521 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:13.521 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:13.521 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:13.592 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:13.586+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:13.602 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:13.603 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:13.603 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:13.603 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:13.603 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:13.612 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:13.693 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:13.686+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:13.718 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:13.718 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.521310256000007, 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.719 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:15:13.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:15:13.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:13.723 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.517440625000006, 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:13.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.727 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.517424558000002, 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.728 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:13.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.517408162999999, 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.732 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:13.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.517398588999995, 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.740 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 85.517389053000002, 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:13.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.744 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 56.819552170000001, 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:13.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:13.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:13.749 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 42.133716915999997, 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:13.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:13.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1991975729999993, 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:13.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1991169020000001, 2024-09-16T11:15:13.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:13.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.766 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1989955099999996, 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:13.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:13.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1989433159999994, 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.772 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1988865830000002, 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:13.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:13.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:13.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1988393839999993, 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:13.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:13.787 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1987894140000002, 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.788 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:13.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:13.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1987223, 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:13.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:13.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1967472140000002, 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:13.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1966987289999995, 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:13.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:13.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1965817009999995, 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:13.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:13.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.1964905259999998, 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:13.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:13.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:13.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:13.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:13.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:13.819 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 1.45686 -> 1.45686 (expected baseline 3.3197) 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 15 -> 15 pgs (1.16667 -> 1.16667 * mean) 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:13.820 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:13.821 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:13.821 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:13.821 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:13.821 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0052 2024-09-16T11:15:13.822 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:13.822 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:13.823 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:13.823 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-16T11:15:13.823 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:13.814+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:13.921 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:13.917+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:13.929 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:13.929 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:13.929 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:13.930 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:13.939 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:14.023 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:14.018+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:14.077 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:14.077 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:14.077 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:14.077 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:14.078 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:14.087 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:14.124 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:14.121+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:14.161 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:14.161 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:14.161 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 1.45686 -> 1.45686 (expected baseline 3.3197) 2024-09-16T11:15:14.161 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:14.162 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 15 -> 15 pgs (1.16667 -> 1.16667 * mean) 2024-09-16T11:15:14.162 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:14.162 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:15:14.162 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 3 2024-09-16T11:15:14.162 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:14.163 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:14.163 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0473 2024-09-16T11:15:14.165 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:14.175 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:14.175 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-16T11:15:14.189 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:14.189 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:14.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:14.189 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:14.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.190 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.190 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:14.190 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.40635974, 2024-09-16T11:15:14.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:14.191 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.191 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.191 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:14.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.192 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:14.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:14.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:14.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:14.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:14.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:14.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01843403999999, 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.203 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:14.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:14.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:14.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:14.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:14.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01828270999999, 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:14.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:14.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:14.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.018231224, 2024-09-16T11:15:14.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:14.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:14.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01816107600001, 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:14.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01798860599999, 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:14.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.239 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01793028399999, 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:14.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01777816399999, 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:14.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:14.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01759964499999, 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:14.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:14.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 144.01756667199999, 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:14.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:14.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:14.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.267 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.465378947, 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.268 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.275 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.46525015, 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.276 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:14.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:14.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:14.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.46519180499999, 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:14.283 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:14.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:14.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:14.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:14.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.290 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.465137416, 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.291 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:14.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:14.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:14.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.298 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.46500310900001, 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:14.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:14.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.305 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 142.68384774, 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.306 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:15:14.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:15:14.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.312 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 142.683760428, 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.313 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:14.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:14.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.319 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 142.68352229999999, 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:14.320 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:14.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:14.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 72.003142291000003, 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:15:14.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:15:14.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.294949854999999, 2024-09-16T11:15:14.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:14.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:14.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:14.338 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:14.338 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:14.339 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:14.340 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:14.334+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:14.440 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:14.434+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:14.441 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:14.441 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:14.441 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:14.441 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:14.450 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:14.470 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:14.470 INFO:teuthology.orchestra.run.smithi162.stdout:avg 12.8571 2024-09-16T11:15:14.470 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 1.45686 -> 1.45686 (expected baseline 3.3197) 2024-09-16T11:15:14.470 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 10 -> 10 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:14.470 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 15 -> 15 pgs (1.16667 -> 1.16667 * mean) 2024-09-16T11:15:14.471 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:14.471 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:14.471 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 3 2024-09-16T11:15:14.471 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:14.471 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0047 2024-09-16T11:15:14.471 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0052 2024-09-16T11:15:14.474 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:14.542 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:14.541+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:14.549 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:14.549 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:14.549 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:14.549 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:14.549 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:14.558 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:14.643 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:14.641+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:14.664 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.485519214999997, 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:14.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.483055941000003, 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.669 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:15:14.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.481847705999996, 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:14.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.479782413999999, 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.479768558000004, 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.682 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:14.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:14.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.479754682999996, 2024-09-16T11:15:14.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:14.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:14.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.690 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.479740915999997, 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:14.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:14.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.694 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.479693406999999, 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:14.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:14.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.479407589000004, 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:14.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:14.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.386743773000006, 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:14.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:14.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.386594591000005, 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.710 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:14.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:14.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:14.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:14.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398193371, 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:14.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:14.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.721 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398146411000001, 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:14.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398127719, 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:14.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:14.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.39811152, 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.731 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:14.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:14.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398096477999999, 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:14.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:14.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398080728, 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:14.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:14.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:14.744 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398065438, 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:14.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.39804986, 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.749 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.753 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.398034880999999, 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:14.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:14.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:14.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:14.758 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:14.760 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:14.760 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:14.757+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:14.861 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:14.857+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:14.864 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:14.864 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:14.864 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:14.864 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:14.873 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:14.962 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:14.961+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:14.969 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:14.969 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:14.969 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:14.969 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:14.969 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:14.978 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:15.062 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:15.061+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.581373646000003, 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.327960848000004, 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.327691532000003, 2024-09-16T11:15:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.327293021999999, 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.327004234, 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.325304817999999, 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.325226502, 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.325192762999997, 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.325152017999997, 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.321951570000003, 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.321933512999998, 2024-09-16T11:15:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:15.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.321907523999997, 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.135 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:15.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.139 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.321209093, 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.140 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.321196534000002, 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:15.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:15.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.411520781, 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:15.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.729095276000001, 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.158 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:15.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:15.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.165 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.726230383000001, 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.166 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:15.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:15.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:15.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:15.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.717184466999999, 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:15.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:15.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.659768961999999, 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.180 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:15.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.613509797000001, 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:15.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.195 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:15.196 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:15.197 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:15.194+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:15.297 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:15.290+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:15.304 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:15.304 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:15.305 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:15.314 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:15.398 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:15.394+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:15.415 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:15.416 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:15.416 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:15.424 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:15.499 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:15.494+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:15.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:15:15.528 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447275955000002, 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447261497, 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447244296999997, 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447230777000001, 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447222234999998, 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447212090999997, 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447198583999999, 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447184991999997, 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:15.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447160709999999, 2024-09-16T11:15:15.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:15.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447126548999996, 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:15.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447093152000001, 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.582 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:15.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:15.587 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447070506999999, 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:15.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:15.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:15.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447054932, 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.593 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:15.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:15.597 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.447037897999998, 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.598 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:15.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:15.603 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.443404934, 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:15.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:15.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:15.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.608 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.443370592000001, 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:15.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:15.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.443354646000003, 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.614 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.616 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:15.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.443339035000001, 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.621 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:15.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:15.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.443323947000003, 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.626 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:15.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.443309145000001, 2024-09-16T11:15:15.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:15.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:15.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.636 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.637 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:15.637 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:15.639 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:15.639 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:15.637+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:15.739 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:15.737+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:15.749 INFO:teuthology.orchestra.run.smithi162.stdout:5121 clone 41 order 19 su 4096 sc 3 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "MOSDPGPushReply(4.14 130/129 [PushReplyOp(4:2895bb47:::rbd_data.11b5f978907f.0000000000000076:f)])", 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:15.826889+0000", 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00039174099999999998, 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00042325000000000002, 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:15.829 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.826889+0000", 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.826889+0000", 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:15.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.826890+0000", 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0440000000000001e-06 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.826898+0000", 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1820000000000006e-06 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.826919+0000", 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0316999999999999e-05 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.826928+0000", 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0250000000000008e-06 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:15.827028+0000", 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100411 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:15.832 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:15:15.833 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:15:15.833 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:15.833 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:15.839 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:15.840 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:15.837+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:15.858 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:15.877 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:15.895 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:15.916 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:15.941 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:15.941 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:15.941 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:15.941+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:15.942 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:15.951 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:15.971 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:15.994 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:15.995 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:15.996 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:16.042 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:16.038+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:16.053 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.053 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.063 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:16.143 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:16.138+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:16.172 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.172 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:16.244 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:16.241+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:16.279 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.279 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.288 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:16.345 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:16.342+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:16.386 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.386 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.395 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:16.446 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:16.445+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:16.494 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.494 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.508 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:15:16.509 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:16.546 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:16.542+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:16.647 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:16.642+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:16.651 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.651 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.660 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:16.748 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:16.745+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:16.758 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.758 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.768 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:16.850 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:16.846+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:16.866 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.866 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:16.950 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:16.949+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:16.981 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:16.981 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:16.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:17.052 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:17.046+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:17.085 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.085 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.094 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:17.151 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:17.146+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:17.190 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.190 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.199 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:17.253 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:17.246+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:17.294 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.294 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.304 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:17.354 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:17.353+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:17.410 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.410 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.419 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:17.462 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:17.461+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:17.533 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.533 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.543 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:17.555 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:17.553+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:17.656 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:17.650+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:17.675 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.675 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.685 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:17.757 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:17.754+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:17.782 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.782 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.791 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:17.857 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:17.857+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:17.864 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-16T11:15:17.959 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:17.957+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:17.967 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:17.967 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:17.971 INFO:teuthology.orchestra.run.smithi162.stdout:5122 read 0x55eb36d thru 0x55f92c7 (0xdf5b bytes) 2024-09-16T11:15:17.977 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:17.979 INFO:teuthology.orchestra.run.smithi162.stdout:5123 write 0x700a608 thru 0x7019ee9 (0xf8e2 bytes) 2024-09-16T11:15:17.992 INFO:teuthology.orchestra.run.smithi162.stdout:5125 punch from 0x26ae5b1 to 0x26bb08f, (0xcade bytes) 2024-09-16T11:15:17.992 INFO:teuthology.orchestra.run.smithi162.stdout:5126 read 0x579a26a thru 0x57a2e9d (0x8c34 bytes) 2024-09-16T11:15:17.993 INFO:teuthology.orchestra.run.smithi162.stdout:5128 writesame 0x7e5bee8 thru 0x7e5f42e data_size 0x1(0x3547 bytes) 2024-09-16T11:15:18.007 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:18.008 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.008 INFO:teuthology.orchestra.run.smithi162.stdout:5131 write 0xd4b3bfc thru 0xd4b6d80 (0x3185 bytes) 2024-09-16T11:15:18.014 INFO:teuthology.orchestra.run.smithi162.stdout:5133 write 0xe7b11d thru 0xe830f3 (0x7fd7 bytes) 2024-09-16T11:15:18.030 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.031 INFO:teuthology.orchestra.run.smithi162.stdout:5136 writesame 0xc7d6acc thru 0xc7df47d data_size 0x1e(0x89b2 bytes) 2024-09-16T11:15:18.049 INFO:teuthology.orchestra.run.smithi162.stdout:5137 read 0x3638c66 thru 0x3648743 (0xfade bytes) 2024-09-16T11:15:18.050 INFO:teuthology.orchestra.run.smithi162.stdout:5138 writesame 0x1b08146 thru 0x1b0e825 data_size 0x10(0x66e0 bytes) 2024-09-16T11:15:18.059 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:18.057+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:18.072 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.072 INFO:teuthology.orchestra.run.smithi162.stdout:5140 read 0x5f52fbc thru 0x5f55b8a (0x2bcf bytes) 2024-09-16T11:15:18.072 INFO:teuthology.orchestra.run.smithi162.stdout:5142 read 0x97e511d thru 0x97e66c2 (0x15a6 bytes) 2024-09-16T11:15:18.072 INFO:teuthology.orchestra.run.smithi162.stdout:5143 read 0x416d2b5 thru 0x417bdb4 (0xeb00 bytes) 2024-09-16T11:15:18.074 INFO:teuthology.orchestra.run.smithi162.stdout:5146 read 0xb46b578 thru 0xb473b93 (0x861c bytes) 2024-09-16T11:15:18.074 INFO:teuthology.orchestra.run.smithi162.stdout:5148 writesame 0xcaf4bb4 thru 0xcb02d27 data_size 0x4(0xe174 bytes) 2024-09-16T11:15:18.094 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:18.094 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:18.096 INFO:teuthology.orchestra.run.smithi162.stdout:5149 write 0x8675588 thru 0x86777ad (0x2226 bytes) 2024-09-16T11:15:18.105 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:18.106 INFO:teuthology.orchestra.run.smithi162.stdout:5150 write 0x935e526 thru 0x93630bc (0x4b97 bytes) 2024-09-16T11:15:18.130 INFO:teuthology.orchestra.run.smithi162.stdout:5151 writesame 0xe07bd35 thru 0xe07dd5d data_size 0x1(0x2029 bytes) 2024-09-16T11:15:18.130 INFO:teuthology.orchestra.run.smithi162.stdout:5153 read 0x582fce0 thru 0x583f9cd (0xfcee bytes) 2024-09-16T11:15:18.131 INFO:teuthology.orchestra.run.smithi162.stdout:5154 read 0x948cb90 thru 0x948d364 (0x7d5 bytes) 2024-09-16T11:15:18.131 INFO:teuthology.orchestra.run.smithi162.stdout:5155 read 0xdf59d8a thru 0xdf68c3b (0xeeb2 bytes) 2024-09-16T11:15:18.131 INFO:teuthology.orchestra.run.smithi162.stdout:5156 read 0x9dabbb5 thru 0x9db4be8 (0x9034 bytes) 2024-09-16T11:15:18.132 INFO:teuthology.orchestra.run.smithi162.stdout:5157 write 0x1a9ee3a thru 0x1aad72b (0xe8f2 bytes) 2024-09-16T11:15:18.144 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:18.156 INFO:teuthology.orchestra.run.smithi162.stdout:5158 write 0x224f732 thru 0x225f462 (0xfd31 bytes) 2024-09-16T11:15:18.160 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:18.157+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:18.169 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:18.186 INFO:teuthology.orchestra.run.smithi162.stdout:5159 trunc from 0xe07dd5e to 0xdad4e5 2024-09-16T11:15:18.203 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:18.220 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:18.220 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:18.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.724803992999995, 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:18.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:18.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:18.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.471391195000002, 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.471121879000002, 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.232 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.470723368999998, 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:18.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:18.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.470434580999999, 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:18.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.468735164999998, 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.245 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.468656848999998, 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.468623110000003, 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:18.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.468582365000003, 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:18.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:18.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:18.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.465381917000002, 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:18.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:18.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.465363859999997, 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:18.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:18.270 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.465337871000003, 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.271 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:18.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:18.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:18.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.464639439999999, 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:18.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.464626881000001, 2024-09-16T11:15:18.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:18.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.554951127999999, 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:18.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:18.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.872525623, 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.293 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:18.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.86966073, 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.300 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:18.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.860614814, 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.307 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:18.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:18.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:18.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.803199309, 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:18.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.756940144, 2024-09-16T11:15:18.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:18.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:18.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.326 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.327 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:18.328 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:18.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:18.329 INFO:teuthology.orchestra.run.smithi162.stdout:5161 write 0x87ba13a thru 0x87c09f9 (0x68c0 bytes) 2024-09-16T11:15:18.329 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:18.330 INFO:teuthology.orchestra.run.smithi162.stdout:5162 writesame 0x3101d80 thru 0x3107a58 data_size 0x39(0x5cd9 bytes) 2024-09-16T11:15:18.330 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:18.331 INFO:teuthology.orchestra.run.smithi162.stdout:5163 write 0xdf8b4c7 thru 0xdf92133 (0x6c6d bytes) 2024-09-16T11:15:18.331 INFO:teuthology.orchestra.run.smithi162.stdout:5164 read 0x720cf1f thru 0x721729f (0xa381 bytes) 2024-09-16T11:15:18.331 INFO:teuthology.orchestra.run.smithi162.stdout:5165 read 0xe1bc88 thru 0xe1f0ed (0x3466 bytes) 2024-09-16T11:15:18.331 INFO:teuthology.orchestra.run.smithi162.stdout:5166 writesame 0x1ce88b3 thru 0x1cee04a data_size 0x4(0x5798 bytes) 2024-09-16T11:15:18.331 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:18.332 INFO:teuthology.orchestra.run.smithi162.stdout:5167 writesame 0x60f5ea0 thru 0x60f8e4c data_size 0x1(0x2fad bytes) 2024-09-16T11:15:18.332 INFO:teuthology.orchestra.run.smithi162.stdout:5168 write 0xd9cac0b thru 0xd9d8697 (0xda8d bytes) 2024-09-16T11:15:18.332 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:18.332 INFO:teuthology.orchestra.run.smithi162.stdout:5169 write 0x79ac0ff thru 0x79b3f2a (0x7e2c bytes) 2024-09-16T11:15:18.332 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:18.333 INFO:teuthology.orchestra.run.smithi162.stdout:5170 writesame 0xd1c2fa3 thru 0xd1ccc9d data_size 0x1(0x9cfb bytes) 2024-09-16T11:15:18.335 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:18.325+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:18.352 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.352 INFO:teuthology.orchestra.run.smithi162.stdout:5172 write 0x792020e thru 0x79272bd (0x70b0 bytes) 2024-09-16T11:15:18.360 INFO:teuthology.orchestra.run.smithi162.stdout:5173 read 0x755b03d thru 0x7560ef7 (0x5ebb bytes) 2024-09-16T11:15:18.360 INFO:teuthology.orchestra.run.smithi162.stdout:5174 punch from 0x46db2c3 to 0x46e39af, (0x86ec bytes) 2024-09-16T11:15:18.361 INFO:teuthology.orchestra.run.smithi162.stdout:5175 write 0x56c4b68 thru 0x56d3d7d (0xf216 bytes) 2024-09-16T11:15:18.369 INFO:teuthology.orchestra.run.smithi162.stdout:5176 read 0xf76cf8 thru 0xf7f928 (0x8c31 bytes) 2024-09-16T11:15:18.369 INFO:teuthology.orchestra.run.smithi162.stdout:5179 read 0x1d7d99d thru 0x1d7ea50 (0x10b4 bytes) 2024-09-16T11:15:18.369 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.370 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.370 INFO:teuthology.orchestra.run.smithi162.stdout:5182 read 0x3ea90c2 thru 0x3eada9d (0x49dc bytes) 2024-09-16T11:15:18.370 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.370 INFO:teuthology.orchestra.run.smithi162.stdout:5184 writesame 0xce7cc24 thru 0xce81c5f data_size 0x4f(0x503c bytes) 2024-09-16T11:15:18.379 INFO:teuthology.orchestra.run.smithi162.stdout:5185 read 0x1db8961 thru 0x1dbfbdb (0x727b bytes) 2024-09-16T11:15:18.385 INFO:teuthology.orchestra.run.smithi162.stdout:5186 trunc from 0xdf92134 to 0xaf83358 2024-09-16T11:15:18.405 INFO:teuthology.orchestra.run.smithi162.stdout:5187 read 0x3e2868b thru 0x3e2c3df (0x3d55 bytes) 2024-09-16T11:15:18.405 INFO:teuthology.orchestra.run.smithi162.stdout:5188 write 0x606449f thru 0x606ecdb (0xa83d bytes) 2024-09-16T11:15:18.409 INFO:teuthology.orchestra.run.smithi162.stdout:5189 read 0x80fd1b2 thru 0x80fd890 (0x6df bytes) 2024-09-16T11:15:18.409 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.409 INFO:teuthology.orchestra.run.smithi162.stdout:5191 write 0xa2d68b7 thru 0xa2e1fdc (0xb726 bytes) 2024-09-16T11:15:18.417 INFO:teuthology.orchestra.run.smithi162.stdout:5192 write 0xa527748 thru 0xa52e96d (0x7226 bytes) 2024-09-16T11:15:18.429 INFO:teuthology.orchestra.run.smithi162.stdout:5193 trunc from 0xaf83358 to 0x8b7b741 2024-09-16T11:15:18.429 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:18.425+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:18.437 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:18.438 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:18.438 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:18.448 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:18.449 INFO:teuthology.orchestra.run.smithi162.stdout:5194 punch from 0x4c96c64 to 0x4ca30b4, (0xc450 bytes) 2024-09-16T11:15:18.449 INFO:teuthology.orchestra.run.smithi162.stdout:5195 read 0x1164a33 thru 0x1171fec (0xd5ba bytes) 2024-09-16T11:15:18.451 INFO:teuthology.orchestra.run.smithi162.stdout:5197 trunc from 0x8b7b741 to 0x8182435 2024-09-16T11:15:18.470 INFO:teuthology.orchestra.run.smithi162.stdout:5198 read 0xea3eb thru 0xf9a37 (0xf64d bytes) 2024-09-16T11:15:18.480 INFO:teuthology.orchestra.run.smithi162.stdout:5199 writesame 0x707799 thru 0x711d53 data_size 0x4d(0xa5bb bytes) 2024-09-16T11:15:18.503 INFO:teuthology.orchestra.run.smithi162.stdout:5201 writesame 0x44a7468 thru 0x44ae51e data_size 0x1d(0x70b7 bytes) 2024-09-16T11:15:18.513 INFO:teuthology.orchestra.run.smithi162.stdout:5202 read 0x4315055 thru 0x431bafb (0x6aa7 bytes) 2024-09-16T11:15:18.513 INFO:teuthology.orchestra.run.smithi162.stdout:5204 read 0x606c09 thru 0x6128c9 (0xbcc1 bytes) 2024-09-16T11:15:18.515 INFO:teuthology.orchestra.run.smithi162.stdout:5205 write 0x4416508 thru 0x4422890 (0xc389 bytes) 2024-09-16T11:15:18.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:15:18.522 INFO:teuthology.orchestra.run.smithi162.stdout:5206 trunc from 0x8182435 to 0x88e15a9 2024-09-16T11:15:18.528 INFO:teuthology.orchestra.run.smithi162.stdout:5207 write 0x29844e9 thru 0x298f54e (0xb066 bytes) 2024-09-16T11:15:18.529 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:18.529+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:18.539 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.539 INFO:teuthology.orchestra.run.smithi162.stdout:5210 read 0x184f70d thru 0x185b1a5 (0xba99 bytes) 2024-09-16T11:15:18.539 INFO:teuthology.orchestra.run.smithi162.stdout:5211 read 0x57f9a37 thru 0x57fb3cd (0x1997 bytes) 2024-09-16T11:15:18.550 INFO:teuthology.orchestra.run.smithi162.stdout:5212 write 0xa08e869 thru 0xa09a0dc (0xb874 bytes) 2024-09-16T11:15:18.556 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:18.556 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:18.556 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:18.556 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:18.556 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:18.561 INFO:teuthology.orchestra.run.smithi162.stdout:5213 writesame 0x620f79e thru 0x621cb73 data_size 0x3a(0xd3d6 bytes) 2024-09-16T11:15:18.567 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:18.571 INFO:teuthology.orchestra.run.smithi162.stdout:5214 read 0x936c5e4 thru 0x9376b35 (0xa552 bytes) 2024-09-16T11:15:18.572 INFO:teuthology.orchestra.run.smithi162.stdout:5216 read 0x36b6124 thru 0x36c25e5 (0xc4c2 bytes) 2024-09-16T11:15:18.586 INFO:teuthology.orchestra.run.smithi162.stdout:5217 write 0xd203ada thru 0xd20a832 (0x6d59 bytes) 2024-09-16T11:15:18.598 INFO:teuthology.orchestra.run.smithi162.stdout:5218 read 0x249f11e thru 0x24a7709 (0x85ec bytes) 2024-09-16T11:15:18.598 INFO:teuthology.orchestra.run.smithi162.stdout:5219 punch from 0x402ef6e to 0x402f646, (0x6d8 bytes) 2024-09-16T11:15:18.599 INFO:teuthology.orchestra.run.smithi162.stdout:5220 read 0x1662877 thru 0x166778b (0x4f15 bytes) 2024-09-16T11:15:18.599 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.599 INFO:teuthology.orchestra.run.smithi162.stdout:5223 writesame 0xbefe39b thru 0xbf0708c data_size 0x1(0x8cf2 bytes) 2024-09-16T11:15:18.625 INFO:teuthology.orchestra.run.smithi162.stdout:5224 writesame 0xd149f1a thru 0xd14f2ba data_size 0x1(0x53a1 bytes) 2024-09-16T11:15:18.630 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:18.626+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:18.640 INFO:teuthology.orchestra.run.smithi162.stdout:5225 write 0x107935d thru 0x1082bcc (0x9870 bytes) 2024-09-16T11:15:18.662 INFO:teuthology.orchestra.run.smithi162.stdout:5226 trunc from 0xd20a833 to 0x7cd2b98 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:18.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597706928000001, 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.684 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597692469999998, 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:18.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.687 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.687 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:18.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.692 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597675270000003, 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:18.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:18.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:18.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.59766175, 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597653207999997, 2024-09-16T11:15:18.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:18.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:18.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597643064000003, 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:18.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:18.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597629556999998, 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.714 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:18.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.719 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597615965000003, 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:18.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:18.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597591682999997, 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:18.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597557522000002, 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:18.735 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597524125, 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:18.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:18.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597501479999998, 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:18.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597485904999999, 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:18.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.597468870999997, 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.593835906999999, 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:18.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.593801565, 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.761 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:18.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.593785619000002, 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:18.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:18.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.593770008, 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.771 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:18.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:18.776 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.593754920000002, 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:18.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:18.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.781 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.593740117999999, 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:18.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:18.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:18.786 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:18.788 INFO:teuthology.orchestra.run.smithi162.stdout:5227 read 0x585df65 thru 0x5866cbc (0x8d58 bytes) 2024-09-16T11:15:18.788 INFO:teuthology.orchestra.run.smithi162.stdout:5229 trunc from 0x7cd2b98 to 0xa2b1b1a 2024-09-16T11:15:18.789 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:5230 punch from 0x8978244 to 0x897e0f0, (0x5eac bytes) 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:5233 write 0xd673597 thru 0xd673b78 (0x5e2 bytes) 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:5234 write 0xbac11ce thru 0xbac381d (0x2650 bytes) 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:5235 write 0xc68713b thru 0xc68da79 (0x693f bytes) 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:5238 write 0xca26d78 thru 0xca347e1 (0xda6a bytes) 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:5240 trunc from 0xd673b79 to 0x19cc880 2024-09-16T11:15:18.790 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.791 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:18.785+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:18.793 INFO:teuthology.orchestra.run.smithi162.stdout:5246 trunc from 0x19cc880 to 0x82bf7fa 2024-09-16T11:15:18.810 INFO:teuthology.orchestra.run.smithi162.stdout:5247 writesame 0x8e47772 thru 0x8e55ef5 data_size 0x42(0xe784 bytes) 2024-09-16T11:15:18.824 INFO:teuthology.orchestra.run.smithi162.stdout:5248 write 0xb89eaf4 thru 0xb8a5899 (0x6da6 bytes) 2024-09-16T11:15:18.834 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:18.834 INFO:teuthology.orchestra.run.smithi162.stdout:5250 read 0x9ec8c6e thru 0x9eccc40 (0x3fd3 bytes) 2024-09-16T11:15:18.848 INFO:teuthology.orchestra.run.smithi162.stdout:5251 write 0xe1a777a thru 0xe1ac48a (0x4d11 bytes) 2024-09-16T11:15:18.863 INFO:teuthology.orchestra.run.smithi162.stdout:5252 read 0x2d4d48e thru 0x2d5b9e8 (0xe55b bytes) 2024-09-16T11:15:18.863 INFO:teuthology.orchestra.run.smithi162.stdout:5256 write 0x60b14d7 thru 0x60b8f4c (0x7a76 bytes) 2024-09-16T11:15:18.870 INFO:teuthology.orchestra.run.smithi162.stdout:5257 punch from 0x56f27d to 0x5748a6, (0x5629 bytes) 2024-09-16T11:15:18.870 INFO:teuthology.orchestra.run.smithi162.stdout:5258 write 0x8d622c1 thru 0x8d69ddc (0x7b1c bytes) 2024-09-16T11:15:18.880 INFO:teuthology.orchestra.run.smithi162.stdout:5259 read 0x22fe8e4 thru 0x230d1e1 (0xe8fe bytes) 2024-09-16T11:15:18.880 INFO:teuthology.orchestra.run.smithi162.stdout:5262 writesame 0xdb8b860 thru 0xdb8f5a4 data_size 0x1(0x3d45 bytes) 2024-09-16T11:15:18.888 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:18.885+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:18.894 INFO:teuthology.orchestra.run.smithi162.stdout:5263 read 0x7df0e60 thru 0x7e00da5 (0xff46 bytes) 2024-09-16T11:15:18.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:18.912 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:18.912 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:18.912 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:18.914 INFO:teuthology.orchestra.run.smithi162.stdout:5264 trunc from 0xe1ac48b to 0x7a5f73 2024-09-16T11:15:18.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:18.950 INFO:teuthology.orchestra.run.smithi162.stdout:5266 read 0x10b21c thru 0x11346e (0x8253 bytes) 2024-09-16T11:15:18.952 INFO:teuthology.orchestra.run.smithi162.stdout:5267 read 0xa3c82 thru 0xab735 (0x7ab4 bytes) 2024-09-16T11:15:18.952 INFO:teuthology.orchestra.run.smithi162.stdout:5268 read 0x12e181 thru 0x13a388 (0xc208 bytes) 2024-09-16T11:15:18.975 INFO:teuthology.orchestra.run.smithi162.stdout:5269 write 0x6f57c07 thru 0x6f67a4b (0xfe45 bytes) 2024-09-16T11:15:18.985 INFO:teuthology.orchestra.run.smithi162.stdout:5271 read 0x3a7364c thru 0x3a7e1aa (0xab5f bytes) 2024-09-16T11:15:18.985 INFO:teuthology.orchestra.run.smithi162.stdout:5275 write 0x6a3320a thru 0x6a4237a (0xf171 bytes) 2024-09-16T11:15:18.989 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:18.986+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:19.018 INFO:teuthology.orchestra.run.smithi162.stdout:5276 write 0xce9156f thru 0xce968a4 (0x5336 bytes) 2024-09-16T11:15:19.040 INFO:teuthology.orchestra.run.smithi162.stdout:5277 trunc from 0xce968a5 to 0x584b590 2024-09-16T11:15:19.065 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:19.065 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:19.065 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:19.065 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:19.065 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:19.072 INFO:teuthology.orchestra.run.smithi162.stdout:5278 trunc from 0x584b590 to 0xd9b280e 2024-09-16T11:15:19.077 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:19.082 INFO:teuthology.orchestra.run.smithi162.stdout:5280 punch from 0xf1fb1f to 0xf28e64, (0x9345 bytes) 2024-09-16T11:15:19.082 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.094 INFO:teuthology.orchestra.run.smithi162.stdout:5282 write 0xe968449 thru 0xe968ac7 (0x67f bytes) 2024-09-16T11:15:19.096 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:19.090+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:19.113 INFO:teuthology.orchestra.run.smithi162.stdout:5283 trunc from 0xe968ac8 to 0x7f9b542 2024-09-16T11:15:19.135 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.135 INFO:teuthology.orchestra.run.smithi162.stdout:5286 read 0x474ca55 thru 0x475872b (0xbcd7 bytes) 2024-09-16T11:15:19.136 INFO:teuthology.orchestra.run.smithi162.stdout:5287 punch from 0x13cfad0 to 0x13d549b, (0x59cb bytes) 2024-09-16T11:15:19.136 INFO:teuthology.orchestra.run.smithi162.stdout:5288 read 0x45481a9 thru 0x45487e5 (0x63d bytes) 2024-09-16T11:15:19.150 INFO:teuthology.orchestra.run.smithi162.stdout:5289 write 0x97bd859 thru 0x97ca47f (0xcc27 bytes) 2024-09-16T11:15:19.162 INFO:teuthology.orchestra.run.smithi162.stdout:5290 write 0x1885ac0 thru 0x188e59b (0x8adc bytes) 2024-09-16T11:15:19.170 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.170 INFO:teuthology.orchestra.run.smithi162.stdout:5292 read 0x59fd8ae thru 0x5a01b68 (0x42bb bytes) 2024-09-16T11:15:19.170 INFO:teuthology.orchestra.run.smithi162.stdout:5294 write 0x3d772 thru 0x3e30e (0xb9d bytes) 2024-09-16T11:15:19.173 INFO:teuthology.orchestra.run.smithi162.stdout:5295 write 0x7e4fe73 thru 0x7e5c88c (0xca1a bytes) 2024-09-16T11:15:19.191 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:19.186+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:19.192 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055880079000005, 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:19.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055861691000004, 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:19.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:19.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055560933999999, 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:19.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055531228999996, 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:19.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055517831000003, 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:19.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:19.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055504026999998, 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:19.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:19.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:19.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.055129027999996, 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:19.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:19.224 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.041584997000001, 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:19.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.041563422999999, 2024-09-16T11:15:19.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.233 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.041546924000002, 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:19.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.041530847999994, 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:19.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.041147273999997, 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:19.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.029065097, 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.252 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:19.252 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.029056896, 2024-09-16T11:15:19.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.253 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:19.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:19.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.029048873999997, 2024-09-16T11:15:19.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:19.259 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.259 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.260 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:19.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:19.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:19.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.345569994999998, 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:19.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:19.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.112151644000001, 2024-09-16T11:15:19.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:19.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:19.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.725758270999997, 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:19.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:19.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:19.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:19.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:19.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.774153622999997, 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:19.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:19.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:19.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:19.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:19.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:19.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:19.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:19.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:19.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:19.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:19.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:19.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:19.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:19.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:19.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:19.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:19.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:19.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:19.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:19.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:19.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:19.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:19.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:19.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:19.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:19.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:19.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:19.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:19.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 38.638356690999998, 2024-09-16T11:15:19.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:19.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:19.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:19.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:19.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:19.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.303 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:19.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.303 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:19.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:19.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:19.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:19.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:19.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:19.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:19.309 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:19.310 INFO:teuthology.orchestra.run.smithi162.stdout:5296 writesame 0xca98504 thru 0xcaa5fe0 data_size 0x1(0xdadd bytes) 2024-09-16T11:15:19.310 INFO:teuthology.orchestra.run.smithi162.stdout:5297 read 0x3de201e thru 0x3deef03 (0xcee6 bytes) 2024-09-16T11:15:19.311 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5298 trunc from 0xcaa5fe1 to 0x9295ffd 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5299 read 0x12dac2b thru 0x12e8803 (0xdbd9 bytes) 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5302 read 0x48cd71 thru 0x493275 (0x6505 bytes) 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5303 read 0x3aae71f thru 0x3aaf594 (0xe76 bytes) 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5304 write 0x8b037f3 thru 0x8b0725c (0x3a6a bytes) 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5305 write 0x68811c0 thru 0x6889dd5 (0x8c16 bytes) 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5310 read 0xd5d30a thru 0xd6980e (0xc505 bytes) 2024-09-16T11:15:19.313 INFO:teuthology.orchestra.run.smithi162.stdout:5311 punch from 0x3ba4265 to 0x3ba5eaa, (0x1c45 bytes) 2024-09-16T11:15:19.314 INFO:teuthology.orchestra.run.smithi162.stdout:5312 write 0x7b4cfa0 thru 0x7b5c95c (0xf9bd bytes) 2024-09-16T11:15:19.314 INFO:teuthology.orchestra.run.smithi162.stdout:5313 read 0x7b01f9a thru 0x7b02c82 (0xce9 bytes) 2024-09-16T11:15:19.314 INFO:teuthology.orchestra.run.smithi162.stdout:5315 writesame 0xcc56e45 thru 0xcc5cf0f data_size 0x1(0x60cb bytes) 2024-09-16T11:15:19.314 INFO:teuthology.orchestra.run.smithi162.stdout:5316 writesame 0xa0b2cf2 thru 0xa0bcde8 data_size 0x1(0xa0f7 bytes) 2024-09-16T11:15:19.314 INFO:teuthology.orchestra.run.smithi162.stdout:5317 writesame 0xbadadd6 thru 0xbae2744 data_size 0x1(0x796f bytes) 2024-09-16T11:15:19.314 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:19.309+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:19.327 INFO:teuthology.orchestra.run.smithi162.stdout:5318 punch from 0xa208bb1 to 0xa20e70c, (0x5b5b bytes) 2024-09-16T11:15:19.327 INFO:teuthology.orchestra.run.smithi162.stdout:5320 write 0x3a3c0be thru 0x3a47088 (0xafcb bytes) 2024-09-16T11:15:19.337 INFO:teuthology.orchestra.run.smithi162.stdout:5322 read 0x1e5ea09 thru 0x1e666cf (0x7cc7 bytes) 2024-09-16T11:15:19.337 INFO:teuthology.orchestra.run.smithi162.stdout:5324 writesame 0x98cebd thru 0x99ce35 data_size 0x1(0xff79 bytes) 2024-09-16T11:15:19.364 INFO:teuthology.orchestra.run.smithi162.stdout:5325 read 0x462ec25 thru 0x4632f8f (0x436b bytes) 2024-09-16T11:15:19.364 INFO:teuthology.orchestra.run.smithi162.stdout:5326 write 0x2ffbf20 thru 0x3007486 (0xb567 bytes) 2024-09-16T11:15:19.381 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.381 INFO:teuthology.orchestra.run.smithi162.stdout:5328 read 0xc490c6 thru 0xc55b0a (0xca45 bytes) 2024-09-16T11:15:19.381 INFO:teuthology.orchestra.run.smithi162.stdout:5329 writesame 0x1d6a795 thru 0x1d6e100 data_size 0x64(0x396c bytes) 2024-09-16T11:15:19.389 INFO:teuthology.orchestra.run.smithi162.stdout:5330 read 0x2cdc9a8 thru 0x2ceae0b (0xe464 bytes) 2024-09-16T11:15:19.389 INFO:teuthology.orchestra.run.smithi162.stdout:5331 write 0x6e8c619 thru 0x6e94ef5 (0x88dd bytes) 2024-09-16T11:15:19.400 INFO:teuthology.orchestra.run.smithi162.stdout:5332 trunc from 0xcc5cf10 to 0xe05d03e 2024-09-16T11:15:19.409 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.409 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.409 INFO:teuthology.orchestra.run.smithi162.stdout:5337 write 0x7413bc8 thru 0x7415a97 (0x1ed0 bytes) 2024-09-16T11:15:19.412 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:19.406+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:193277 4.c 4:30493cec:::rbd_data.11b55ebb7981.00000000000000e8:head [set-alloc-hint object_size 524288 write_size 524288,write 200704~4096 in=4096b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:19.415788+0000", 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.00046763600000000001, 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053889499999999996, 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 193277 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415788+0000", 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415788+0000", 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415790+0000", 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.615e-06 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415803+0000", 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3132999999999999e-05 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415823+0000", 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9958000000000001e-05 2024-09-16T11:15:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415832+0000", 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8959999999999992e-06 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.415891+0000", 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9305000000000002e-05 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.416069+0000", 2024-09-16T11:15:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000177892 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.416215+0000", 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014641799999999999 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:193275 4.c 4:303fb011:::rbd_object_map.11b55ebb7981:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:19.409918+0000", 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.006337489, 2024-09-16T11:15:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00654873, 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 193275 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.409918+0000", 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.409918+0000", 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.409919+0000", 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3200000000000005e-07 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.409922+0000", 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.224e-06 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.409939+0000", 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6424000000000001e-05 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.409943+0000", 2024-09-16T11:15:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0940000000000001e-06 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.412611+0000", 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002668461 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.412747+0000", 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013599900000000001 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.413280+0000", 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053242000000000003 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:19.414620+0000", 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001340686 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:15:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 2 2024-09-16T11:15:19.427 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:19.427 INFO:teuthology.orchestra.run.smithi162.stdout:5339 write 0x46049f thru 0x46ef63 (0xeac5 bytes) 2024-09-16T11:15:19.428 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:19.453 INFO:teuthology.orchestra.run.smithi162.stdout:5340 punch from 0x46d229f to 0x46df0e0, (0xce41 bytes) 2024-09-16T11:15:19.453 INFO:teuthology.orchestra.run.smithi162.stdout:5343 writesame 0xa4a3170 thru 0xa4b1307 data_size 0x4(0xe198 bytes) 2024-09-16T11:15:19.464 INFO:teuthology.orchestra.run.smithi162.stdout:5344 punch from 0x2b05bfd to 0x2b0c381, (0x6784 bytes) 2024-09-16T11:15:19.464 INFO:teuthology.orchestra.run.smithi162.stdout:5346 write 0xbcc8c85 thru 0xbcd1e05 (0x9181 bytes) 2024-09-16T11:15:19.472 INFO:teuthology.orchestra.run.smithi162.stdout:5347 punch from 0x87704be to 0x877f996, (0xf4d8 bytes) 2024-09-16T11:15:19.473 INFO:teuthology.orchestra.run.smithi162.stdout:5348 trunc from 0xe05d03e to 0xe07b968 2024-09-16T11:15:19.482 INFO:teuthology.orchestra.run.smithi162.stdout:5349 write 0x1771d2e thru 0x177a0d4 (0x83a7 bytes) 2024-09-16T11:15:19.485 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 6, 7, 2, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:19.485 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T11:15:19.507 INFO:teuthology.orchestra.run.smithi162.stdout:5350 trunc from 0xe07b968 to 0x7b8b15e 2024-09-16T11:15:19.513 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:19.510+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:19.530 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:19.530 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:19.530 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:19.530 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:19.530 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:19.534 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:15:19.539 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:19.540 INFO:teuthology.orchestra.run.smithi162.stdout:5352 punch from 0x2cb99d1 to 0x2cc7b31, (0xe160 bytes) 2024-09-16T11:15:19.540 INFO:teuthology.orchestra.run.smithi162.stdout:5353 read 0x686fb63 thru 0x6871e25 (0x22c3 bytes) 2024-09-16T11:15:19.549 INFO:teuthology.orchestra.run.smithi162.stdout:5356 trunc from 0x7b8b15e to 0xcdeca77 2024-09-16T11:15:19.557 INFO:teuthology.orchestra.run.smithi162.stdout:5357 read 0x60cbcaa thru 0x60d1ad5 (0x5e2c bytes) 2024-09-16T11:15:19.557 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.557 INFO:teuthology.orchestra.run.smithi162.stdout:5359 read 0x673f779 thru 0x674a885 (0xb10d bytes) 2024-09-16T11:15:19.558 INFO:teuthology.orchestra.run.smithi162.stdout:5361 punch from 0x589d7f0 to 0x58ad327, (0xfb37 bytes) 2024-09-16T11:15:19.558 INFO:teuthology.orchestra.run.smithi162.stdout:5364 writesame 0x83df281 thru 0x83e2bcf data_size 0x1(0x394f bytes) 2024-09-16T11:15:19.575 INFO:teuthology.orchestra.run.smithi162.stdout:5365 punch from 0xe0704e to 0xe10d1b, (0x9ccd bytes) 2024-09-16T11:15:19.575 INFO:teuthology.orchestra.run.smithi162.stdout:5366 read 0xc77db8e thru 0xc786582 (0x89f5 bytes) 2024-09-16T11:15:19.575 INFO:teuthology.orchestra.run.smithi162.stdout:5367 write 0xadf7ee3 thru 0xadf9ef2 (0x2010 bytes) 2024-09-16T11:15:19.584 INFO:teuthology.orchestra.run.smithi162.stdout:5368 write 0xbf6c0bf thru 0xbf7623f (0xa181 bytes) 2024-09-16T11:15:19.593 INFO:teuthology.orchestra.run.smithi162.stdout:5369 punch from 0xa3e36d4 to 0xa3ea4ed, (0x6e19 bytes) 2024-09-16T11:15:19.593 INFO:teuthology.orchestra.run.smithi162.stdout:5370 write 0x4580756 thru 0x45879f6 (0x72a1 bytes) 2024-09-16T11:15:19.601 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.601 INFO:teuthology.orchestra.run.smithi162.stdout:5373 read 0x9e06b52 thru 0x9e0bf50 (0x53ff bytes) 2024-09-16T11:15:19.601 INFO:teuthology.orchestra.run.smithi162.stdout:5374 write 0xb45b5a1 thru 0xb467349 (0xbda9 bytes) 2024-09-16T11:15:19.609 INFO:teuthology.orchestra.run.smithi162.stdout:5375 write 0x1445946 thru 0x1449f51 (0x460c bytes) 2024-09-16T11:15:19.613 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:19.610+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:19.615 INFO:teuthology.orchestra.run.smithi162.stdout:5376 write 0x56c4288 thru 0x56ca7f0 (0x6569 bytes) 2024-09-16T11:15:19.624 INFO:teuthology.orchestra.run.smithi162.stdout:5377 punch from 0xbc5ae96 to 0xbc615cf, (0x6739 bytes) 2024-09-16T11:15:19.644 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567167322000003, 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.645 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:19.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:19.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.650 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567126559999998, 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.651 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:19.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567095762000001, 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:19.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:19.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:19.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567077173999998, 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.662 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:19.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:19.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567059122000003, 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.669 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:19.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:19.674 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567039195, 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:19.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.679 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.567020356, 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.684 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.566828407000003, 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.689 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.566809126999999, 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.694 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563317558999998, 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:19.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.699 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563307488, 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.700 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:19.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563279184999999, 2024-09-16T11:15:19.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:19.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:19.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563265256000001, 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:19.710 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:19.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:19.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563256492999997, 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:19.715 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:19.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563232673999998, 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:19.720 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563201552000002, 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.725 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:19.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:19.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563153813, 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.730 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:19.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:19.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.563131705000004, 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:19.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:19.739 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1539373409999998, 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:19.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:19.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:19.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:19.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:19.746 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 6.1538913610000003, 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:19.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:19.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:19.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:19.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:19.755 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:19.757 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:19.758 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:19.753+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:19.758 INFO:teuthology.orchestra.run.smithi162.stdout:5378 trunc from 0xcdeca77 to 0x8e901 2024-09-16T11:15:19.758 INFO:teuthology.orchestra.run.smithi162.stdout:5379 read 0x802e7 thru 0x833aa (0x30c4 bytes) 2024-09-16T11:15:19.758 INFO:teuthology.orchestra.run.smithi162.stdout:5380 trunc from 0x8e901 to 0xa18875d 2024-09-16T11:15:19.758 INFO:teuthology.orchestra.run.smithi162.stdout:5381 punch from 0x9c289aa to 0x9c31136, (0x878c bytes) 2024-09-16T11:15:19.758 INFO:teuthology.orchestra.run.smithi162.stdout:5382 trunc from 0xa18875d to 0xbd67326 2024-09-16T11:15:19.759 INFO:teuthology.orchestra.run.smithi162.stdout:5383 trunc from 0xbd67326 to 0xc30f1c4 2024-09-16T11:15:19.759 INFO:teuthology.orchestra.run.smithi162.stdout:5384 trunc from 0xc30f1c4 to 0x1364518 2024-09-16T11:15:19.765 INFO:teuthology.orchestra.run.smithi162.stdout:5385 read 0xe0427c thru 0xe0ec8c (0xaa11 bytes) 2024-09-16T11:15:19.765 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.785 INFO:teuthology.orchestra.run.smithi162.stdout:5388 write 0x8e452b7 thru 0x8e52c7c (0xd9c6 bytes) 2024-09-16T11:15:19.812 INFO:teuthology.orchestra.run.smithi162.stdout:5390 write 0xc18ffa6 thru 0xc19b185 (0xb1e0 bytes) 2024-09-16T11:15:19.821 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.822 INFO:teuthology.orchestra.run.smithi162.stdout:5394 read 0xa46c15a thru 0xa470911 (0x47b8 bytes) 2024-09-16T11:15:19.826 INFO:teuthology.orchestra.run.smithi162.stdout:5396 trunc from 0xc19b186 to 0xe3a8106 2024-09-16T11:15:19.845 INFO:teuthology.orchestra.run.smithi162.stdout:5397 trunc from 0xe3a8106 to 0x88e6aef 2024-09-16T11:15:19.857 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:19.854+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:19.867 INFO:teuthology.orchestra.run.smithi162.stdout:5398 read 0x2ac1684 thru 0x2ac362f (0x1fac bytes) 2024-09-16T11:15:19.867 INFO:teuthology.orchestra.run.smithi162.stdout:5399 write 0x644b9f6 thru 0x64537bb (0x7dc6 bytes) 2024-09-16T11:15:19.872 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:19.872 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:19.877 INFO:teuthology.orchestra.run.smithi162.stdout:5400 punch from 0x68c07bb to 0x68ca283, (0x9ac8 bytes) 2024-09-16T11:15:19.877 INFO:teuthology.orchestra.run.smithi162.stdout:5401 write 0x11cff25 thru 0x11dcfaa (0xd086 bytes) 2024-09-16T11:15:19.884 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:19.888 INFO:teuthology.orchestra.run.smithi162.stdout:5403 read 0x2c9fe43 thru 0x2ca03cf (0x58d bytes) 2024-09-16T11:15:19.903 INFO:teuthology.orchestra.run.smithi162.stdout:5404 trunc from 0x88e6aef to 0x1544bd9 2024-09-16T11:15:19.950 INFO:teuthology.orchestra.run.smithi162.stdout:5405 write 0x9de4342 thru 0x9defb93 (0xb852 bytes) 2024-09-16T11:15:19.958 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:19.954+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:19.961 INFO:teuthology.orchestra.run.smithi162.stdout:5406 read 0x6e1c24b thru 0x6e2b33c (0xf0f2 bytes) 2024-09-16T11:15:19.961 INFO:teuthology.orchestra.run.smithi162.stdout:5407 read 0x5bca230 thru 0x5bcf97c (0x574d bytes) 2024-09-16T11:15:19.961 INFO:teuthology.orchestra.run.smithi162.stdout:5408 read 0x4782e68 thru 0x478e079 (0xb212 bytes) 2024-09-16T11:15:19.961 INFO:teuthology.orchestra.run.smithi162.stdout:5409 read 0x823a46a thru 0x824467e (0xa215 bytes) 2024-09-16T11:15:19.961 INFO:teuthology.orchestra.run.smithi162.stdout:5411 punch from 0x4454671 to 0x445ae99, (0x6828 bytes) 2024-09-16T11:15:19.973 INFO:teuthology.orchestra.run.smithi162.stdout:5412 write 0xa730dbd thru 0xa73b86e (0xaab2 bytes) 2024-09-16T11:15:19.982 INFO:teuthology.orchestra.run.smithi162.stdout:5413 writesame 0x83dbc9f thru 0x83e9dc8 data_size 0x27(0xe12a bytes) 2024-09-16T11:15:19.986 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:19.987 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:19.987 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:19.987 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:19.987 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:5415 read 0x689644a thru 0x689953a (0x30f1 bytes) 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:5418 punch from 0x11f11e1 to 0x11f5e98, (0x4cb7 bytes) 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:5419 punch from 0xa6fb868 to 0xa6fed47, (0x34df bytes) 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:19.993 INFO:teuthology.orchestra.run.smithi162.stdout:5428 writesame 0x26a2fec thru 0x26aa984 data_size 0x1(0x7999 bytes) 2024-09-16T11:15:19.996 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:20.014 INFO:teuthology.orchestra.run.smithi162.stdout:5429 read 0x15d178c thru 0x15daf06 (0x977b bytes) 2024-09-16T11:15:20.014 INFO:teuthology.orchestra.run.smithi162.stdout:5430 read 0xffd8ea thru 0x1004886 (0x6f9d bytes) 2024-09-16T11:15:20.024 INFO:teuthology.orchestra.run.smithi162.stdout:5431 trunc from 0xa73b86f to 0x5d8bf82 2024-09-16T11:15:20.044 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.058 INFO:teuthology.orchestra.run.smithi162.stdout:5433 trunc from 0x5d8bf82 to 0x129e79 2024-09-16T11:15:20.059 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:20.057+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:20.096 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 119.22881483800001, 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:20.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.101 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060843188999996, 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:20.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:20.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060831628000003, 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.106 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:20.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060818425999997, 2024-09-16T11:15:20.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:20.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.114 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060805986000005, 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.115 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060792475, 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.119 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:20.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:20.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060781559999995, 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:20.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:20.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060755642000004, 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:20.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.132 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060740187999997, 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060730301999996, 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:20.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:20.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060720539000002, 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:20.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.145 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060710361999995, 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:20.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060693880000002, 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060679183000005, 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.154 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:20.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060660740000003, 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:20.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:20.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060643791000004, 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:20.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:20.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060624578000002, 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:20.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.172 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060607361999999, 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:20.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:20.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060586107999995, 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.177 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:20.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.060566140000006, 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:20.181 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:20.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.185 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.186 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.186 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:20.188 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:20.189 INFO:teuthology.orchestra.run.smithi162.stdout:5435 write 0xcb95e65 thru 0xcb99e97 (0x4033 bytes) 2024-09-16T11:15:20.189 INFO:teuthology.orchestra.run.smithi162.stdout:5437 writesame 0x7b9f45b thru 0x7ba8024 data_size 0x1d(0x8bca bytes) 2024-09-16T11:15:20.189 INFO:teuthology.orchestra.run.smithi162.stdout:5439 read 0x4bbc845 thru 0x4bbcfcc (0x788 bytes) 2024-09-16T11:15:20.189 INFO:teuthology.orchestra.run.smithi162.stdout:5440 write 0x1e38542 thru 0x1e3c516 (0x3fd5 bytes) 2024-09-16T11:15:20.189 INFO:teuthology.orchestra.run.smithi162.stdout:5441 punch from 0xb27073f to 0xb27e8d7, (0xe198 bytes) 2024-09-16T11:15:20.189 INFO:teuthology.orchestra.run.smithi162.stdout:5443 punch from 0x5d99fc to 0x5e9280, (0xf884 bytes) 2024-09-16T11:15:20.190 INFO:teuthology.orchestra.run.smithi162.stdout:5445 read 0x5420c7c thru 0x5420e0e (0x193 bytes) 2024-09-16T11:15:20.190 INFO:teuthology.orchestra.run.smithi162.stdout:5446 write 0xa2018c5 thru 0xa202560 (0xc9c bytes) 2024-09-16T11:15:20.190 INFO:teuthology.orchestra.run.smithi162.stdout:5448 write 0x48d13ea thru 0x48d7062 (0x5c79 bytes) 2024-09-16T11:15:20.190 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.190 INFO:teuthology.orchestra.run.smithi162.stdout:5450 read 0x18d822c thru 0x18e49f0 (0xc7c5 bytes) 2024-09-16T11:15:20.190 INFO:teuthology.orchestra.run.smithi162.stdout:5453 trunc from 0xcb99e98 to 0x6ca27d0 2024-09-16T11:15:20.190 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:20.182+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:20.203 INFO:teuthology.orchestra.run.smithi162.stdout:5454 read 0x568bf92 thru 0x568fb62 (0x3bd1 bytes) 2024-09-16T11:15:20.204 INFO:teuthology.orchestra.run.smithi162.stdout:5455 read 0x2b89324 thru 0x2b8b944 (0x2621 bytes) 2024-09-16T11:15:20.204 INFO:teuthology.orchestra.run.smithi162.stdout:5456 read 0xbe2309 thru 0xbea2df (0x7fd7 bytes) 2024-09-16T11:15:20.204 INFO:teuthology.orchestra.run.smithi162.stdout:5457 read 0x12c0b0b thru 0x12c8546 (0x7a3c bytes) 2024-09-16T11:15:20.222 INFO:teuthology.orchestra.run.smithi162.stdout:5458 writesame 0x9d4d20c thru 0x9d4dd02 data_size 0x1(0xaf7 bytes) 2024-09-16T11:15:20.222 INFO:teuthology.orchestra.run.smithi162.stdout:5459 punch from 0x1aff2a4 to 0x1b0ec8e, (0xf9ea bytes) 2024-09-16T11:15:20.222 INFO:teuthology.orchestra.run.smithi162.stdout:5460 read 0x57d0a79 thru 0x57d93c1 (0x8949 bytes) 2024-09-16T11:15:20.222 INFO:teuthology.orchestra.run.smithi162.stdout:5462 writesame 0x54a043a thru 0x54aff1e data_size 0x1(0xfae5 bytes) 2024-09-16T11:15:20.256 INFO:teuthology.orchestra.run.smithi162.stdout:5463 write 0x55efd0f thru 0x55fe13a (0xe42c bytes) 2024-09-16T11:15:20.265 INFO:teuthology.orchestra.run.smithi162.stdout:5465 writesame 0x35b40b6 thru 0x35b40ca data_size 0x1(0x15 bytes) 2024-09-16T11:15:20.271 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.284 INFO:teuthology.orchestra.run.smithi162.stdout:5467 trunc from 0x9d4dd03 to 0x41a4ee5 2024-09-16T11:15:20.290 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:20.282+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:20.296 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:20.296 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:20.306 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:20.307 INFO:teuthology.orchestra.run.smithi162.stdout:5468 write 0x40ed35c thru 0x40f3c74 (0x6919 bytes) 2024-09-16T11:15:20.316 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.316 INFO:teuthology.orchestra.run.smithi162.stdout:5470 read 0x324f394 thru 0x325b7fb (0xc468 bytes) 2024-09-16T11:15:20.335 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:20.344 INFO:teuthology.orchestra.run.smithi162.stdout:5471 write 0xd7fb6e5 thru 0xd804b71 (0x948d bytes) 2024-09-16T11:15:20.358 INFO:teuthology.orchestra.run.smithi162.stdout:5472 write 0x4e848d8 thru 0x4e85a7a (0x11a3 bytes) 2024-09-16T11:15:20.364 INFO:teuthology.orchestra.run.smithi162.stdout:5474 writesame 0x3e1a992 thru 0x3e23920 data_size 0xd(0x8f8f bytes) 2024-09-16T11:15:20.373 INFO:teuthology.orchestra.run.smithi162.stdout:5475 writesame 0xaef1d83 thru 0xaef39cb data_size 0x1(0x1c49 bytes) 2024-09-16T11:15:20.382 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.382 INFO:teuthology.orchestra.run.smithi162.stdout:5480 writesame 0xad56 thru 0x18c0c data_size 0x69(0xdeb7 bytes) 2024-09-16T11:15:20.388 INFO:teuthology.orchestra.run.smithi162.stdout:5481 write 0x772bdd thru 0x773f74 (0x1398 bytes) 2024-09-16T11:15:20.388 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:20.382+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:20.395 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.395 INFO:teuthology.orchestra.run.smithi162.stdout:5484 read 0xa3e72ac thru 0xa3ef62e (0x8383 bytes) 2024-09-16T11:15:20.406 INFO:teuthology.orchestra.run.smithi162.stdout:5485 trunc from 0xd804b72 to 0x768009b 2024-09-16T11:15:20.411 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:20.412 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:20.412 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:20.412 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:20.412 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:20.422 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:20.438 INFO:teuthology.orchestra.run.smithi162.stdout:5486 write 0xa1e0956 thru 0xa1eb1fd (0xa8a8 bytes) 2024-09-16T11:15:20.469 INFO:teuthology.orchestra.run.smithi162.stdout:5487 writesame 0xecc4fd0 thru 0xecd22b8 data_size 0x1(0xd2e9 bytes) 2024-09-16T11:15:20.473 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:20.475 INFO:teuthology.orchestra.run.smithi162.stdout:5488 trunc from 0xecd22b9 to 0xbb60214 2024-09-16T11:15:20.484 INFO:teuthology.orchestra.run.smithi162.stdout:5489 read 0xb6fccae thru 0xb70941a (0xc76d bytes) 2024-09-16T11:15:20.490 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:20.486+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:20.495 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:20.498 INFO:teuthology.orchestra.run.smithi162.stdout:5490 trunc from 0xbb60214 to 0x40ffcb6 2024-09-16T11:15:20.518 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:20.524 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:20.524 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:20.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:20.524 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:20.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.327923089999999, 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:15:20.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.324053458999998, 2024-09-16T11:15:20.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:20.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.534 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.324037391999994, 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:20.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:20.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:20.539 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.324020997000005, 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:20.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.324011423000002, 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.544 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.324001886999994, 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.548 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:20.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.626165004000001, 2024-09-16T11:15:20.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:20.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.556 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.940329749999997, 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.557 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:20.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:20.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:20.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005810407, 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.564 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:20.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:20.568 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005729735999999, 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.569 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:20.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005608344000001, 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.574 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.575 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.577 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:20.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.00555615, 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:20.579 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:20.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:20.584 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005499416999999, 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:20.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005452218, 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:20.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:20.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:20.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005402247999999, 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.595 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.005335133999999, 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:20.600 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:20.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.605 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.003360047999999, 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.606 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.607 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:20.608 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.610 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.003311563, 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:20.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:20.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.003194535, 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:20.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:20.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.621 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 15.003103360000001, 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:20.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:20.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:20.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:20.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:20.627 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:20.628 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T11:15:20.628 INFO:teuthology.orchestra.run.smithi162.stdout:5491 trunc from 0x40ffcb6 to 0xc5f6220 2024-09-16T11:15:20.629 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:20.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:20.630 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T11:15:20.631 INFO:teuthology.orchestra.run.smithi162.stdout:5492 read 0x2459bd4 thru 0x2460dd3 (0x7200 bytes) 2024-09-16T11:15:20.631 INFO:teuthology.orchestra.run.smithi162.stdout:5493 punch from 0xa629000 to 0xa635991, (0xc991 bytes) 2024-09-16T11:15:20.631 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:20.631 INFO:teuthology.orchestra.run.smithi162.stdout:5494 trunc from 0xc5f6220 to 0x2de479a 2024-09-16T11:15:20.632 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:20.632 INFO:teuthology.orchestra.run.smithi162.stdout:5495 write 0xa503d1d thru 0xa508f42 (0x5226 bytes) 2024-09-16T11:15:20.632 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:20.632 INFO:teuthology.orchestra.run.smithi162.stdout:5497 write 0x5c07e9b thru 0x5c1080d (0x8973 bytes) 2024-09-16T11:15:20.633 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:20.622+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:20.636 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:20.638 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:20.638 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:20.639 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:20.639 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:20.639 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:20.639 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:20.639 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:20.639 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:20.645 INFO:teuthology.orchestra.run.smithi162.stdout:5498 writesame 0xd8907ef thru 0xd89fb8b data_size 0x1(0xf39d bytes) 2024-09-16T11:15:20.645 INFO:teuthology.orchestra.run.smithi162.stdout:5500 write 0x229ea73 thru 0x22a5e07 (0x7395 bytes) 2024-09-16T11:15:20.656 INFO:teuthology.orchestra.run.smithi162.stdout:5502 write 0xd7b7143 thru 0xd7bc54a (0x5408 bytes) 2024-09-16T11:15:20.667 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:20.685 INFO:teuthology.orchestra.run.smithi162.stdout:5505 trunc from 0xd89fb8c to 0x3ba8a9d 2024-09-16T11:15:20.721 INFO:teuthology.orchestra.run.smithi162.stdout:5506 write 0x84cb8a4 thru 0x84d6639 (0xad96 bytes) 2024-09-16T11:15:20.729 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:20.726+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:20.733 INFO:teuthology.orchestra.run.smithi162.stdout:5509 read 0x27fcc99 thru 0x2804881 (0x7be9 bytes) 2024-09-16T11:15:20.746 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:20.746 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:20.750 INFO:teuthology.orchestra.run.smithi162.stdout:5510 write 0xe0f8eaf thru 0xe1055e3 (0xc735 bytes) 2024-09-16T11:15:20.757 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:20.765 INFO:teuthology.orchestra.run.smithi162.stdout:5511 read 0x328e867 thru 0x32947b9 (0x5f53 bytes) 2024-09-16T11:15:20.765 INFO:teuthology.orchestra.run.smithi162.stdout:5512 write 0xb15555f thru 0xb160304 (0xada6 bytes) 2024-09-16T11:15:20.776 INFO:teuthology.orchestra.run.smithi162.stdout:5515 write 0x361eea7 thru 0x3629d5f (0xaeb9 bytes) 2024-09-16T11:15:20.786 INFO:teuthology.orchestra.run.smithi162.stdout:5517 writesame 0x6748707 thru 0x674cca1 data_size 0x1(0x459b bytes) 2024-09-16T11:15:20.805 INFO:teuthology.orchestra.run.smithi162.stdout:5518 writesame 0xd27a4f6 thru 0xd287d80 data_size 0x1(0xd88b bytes) 2024-09-16T11:15:20.831 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:20.826+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:20.840 INFO:teuthology.orchestra.run.smithi162.stdout:5519 write 0xcf27417 thru 0xcf30930 (0x951a bytes) 2024-09-16T11:15:20.849 INFO:teuthology.orchestra.run.smithi162.stdout:5520 read 0xacd9c6 thru 0xad7707 (0x9d42 bytes) 2024-09-16T11:15:20.849 INFO:teuthology.orchestra.run.smithi162.stdout:5521 write 0xa9b97c4 thru 0xa9bfbea (0x6427 bytes) 2024-09-16T11:15:20.857 INFO:teuthology.orchestra.run.smithi162.stdout:5523 write 0xd2a5db8 thru 0xd2b4fcc (0xf215 bytes) 2024-09-16T11:15:20.860 INFO:teuthology.orchestra.run.smithi162.stdout:5525 read 0x1044f13 thru 0x104d90e (0x89fc bytes) 2024-09-16T11:15:20.860 INFO:teuthology.orchestra.run.smithi162.stdout:5528 read 0xd7396c3 thru 0xd7405eb (0x6f29 bytes) 2024-09-16T11:15:20.860 INFO:teuthology.orchestra.run.smithi162.stdout:5529 writesame 0xafc4b16 thru 0xafc90fd data_size 0x4(0x45e8 bytes) 2024-09-16T11:15:20.860 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:20.861 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:20.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:20.870 INFO:teuthology.orchestra.run.smithi162.stdout:5530 writesame 0xbb4a6e9 thru 0xbb596ef data_size 0x1(0xf007 bytes) 2024-09-16T11:15:20.897 INFO:teuthology.orchestra.run.smithi162.stdout:5531 read 0x462dd0b thru 0x463159e (0x3894 bytes) 2024-09-16T11:15:20.897 INFO:teuthology.orchestra.run.smithi162.stdout:5532 punch from 0x91ea9bf to 0x91f40e3, (0x9724 bytes) 2024-09-16T11:15:20.897 INFO:teuthology.orchestra.run.smithi162.stdout:5533 writesame 0x589a847 thru 0x58a76b1 data_size 0x1(0xce6b bytes) 2024-09-16T11:15:20.916 INFO:teuthology.orchestra.run.smithi162.stdout:5534 write 0x50c12cd thru 0x50c12e6 (0x1a bytes) 2024-09-16T11:15:20.931 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:20.929+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:20.936 INFO:teuthology.orchestra.run.smithi162.stdout:5535 trunc from 0xe1055e4 to 0x67ff32b 2024-09-16T11:15:20.956 INFO:teuthology.orchestra.run.smithi162.stdout:5536 read 0x30d291a thru 0x30e0d77 (0xe45e bytes) 2024-09-16T11:15:20.957 INFO:teuthology.orchestra.run.smithi162.stdout:5537 write 0x39f9163 thru 0x3a03e8f (0xad2d bytes) 2024-09-16T11:15:20.966 INFO:teuthology.orchestra.run.smithi162.stdout:5538 write 0x36855f6 thru 0x368bd34 (0x673f bytes) 2024-09-16T11:15:20.973 INFO:teuthology.orchestra.run.smithi162.stdout:5539 trunc from 0x67ff32b to 0x77f1b65 2024-09-16T11:15:20.977 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:20.977 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:20.987 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:20.993 INFO:teuthology.orchestra.run.smithi162.stdout:5540 trunc from 0x77f1b65 to 0x23fd678 2024-09-16T11:15:21.019 INFO:teuthology.orchestra.run.smithi162.stdout:5541 punch from 0x70762a to 0x70bf6f, (0x4945 bytes) 2024-09-16T11:15:21.033 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:21.030+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:21.041 INFO:teuthology.orchestra.run.smithi162.stdout:5542 write 0x98dc8de thru 0x98e9a3d (0xd160 bytes) 2024-09-16T11:15:21.052 INFO:teuthology.orchestra.run.smithi162.stdout:5543 write 0x14168b6 thru 0x1420968 (0xa0b3 bytes) 2024-09-16T11:15:21.063 INFO:teuthology.orchestra.run.smithi162.stdout:5544 read 0x986eb8b thru 0x986fe75 (0x12eb bytes) 2024-09-16T11:15:21.064 INFO:teuthology.orchestra.run.smithi162.stdout:5545 write 0x8ac29fe thru 0x8ac73f1 (0x49f4 bytes) 2024-09-16T11:15:21.086 INFO:teuthology.orchestra.run.smithi162.stdout:5546 write 0xb414669 thru 0xb4156e6 (0x107e bytes) 2024-09-16T11:15:21.094 INFO:teuthology.orchestra.run.smithi162.stdout:5547 write 0x5ae2170 thru 0x5af1b47 (0xf9d8 bytes) 2024-09-16T11:15:21.095 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.095 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.104 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:21.105 INFO:teuthology.orchestra.run.smithi162.stdout:5548 punch from 0xab98339 to 0xab9eb06, (0x67cd bytes) 2024-09-16T11:15:21.105 INFO:teuthology.orchestra.run.smithi162.stdout:5549 read 0xa475667 thru 0xa47c7da (0x7174 bytes) 2024-09-16T11:15:21.118 INFO:teuthology.orchestra.run.smithi162.stdout:5551 write 0xce9d834 thru 0xcea4922 (0x70ef bytes) 2024-09-16T11:15:21.128 INFO:teuthology.orchestra.run.smithi162.stdout:5552 read 0x7d9be thru 0x8014b (0x278e bytes) 2024-09-16T11:15:21.130 INFO:teuthology.orchestra.run.smithi162.stdout:5553 punch from 0xc6f99c7 to 0xc6fc42f, (0x2a68 bytes) 2024-09-16T11:15:21.130 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.130 INFO:teuthology.orchestra.run.smithi162.stdout:5555 writesame 0x9d29bef thru 0x9d3785b data_size 0x1(0xdc6d bytes) 2024-09-16T11:15:21.134 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:21.130+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:21.155 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.155 INFO:teuthology.orchestra.run.smithi162.stdout:5558 punch from 0x1e72cc3 to 0x1e7d66b, (0xa9a8 bytes) 2024-09-16T11:15:21.155 INFO:teuthology.orchestra.run.smithi162.stdout:5559 read 0x49af43 thru 0x4a1f6a (0x7028 bytes) 2024-09-16T11:15:21.155 INFO:teuthology.orchestra.run.smithi162.stdout:5560 writesame 0xca4f78 thru 0xcb4565 data_size 0x15(0xf5ee bytes) 2024-09-16T11:15:21.166 INFO:teuthology.orchestra.run.smithi162.stdout:5561 read 0x8a6ebed thru 0x8a7b4e8 (0xc8fc bytes) 2024-09-16T11:15:21.167 INFO:teuthology.orchestra.run.smithi162.stdout:5563 punch from 0xc093a32 to 0xc099d36, (0x6304 bytes) 2024-09-16T11:15:21.167 INFO:teuthology.orchestra.run.smithi162.stdout:5565 write 0x2446053 thru 0x244a260 (0x420e bytes) 2024-09-16T11:15:21.187 INFO:teuthology.orchestra.run.smithi162.stdout:5567 trunc from 0xcea4923 to 0x55f15bc 2024-09-16T11:15:21.219 INFO:teuthology.orchestra.run.smithi162.stdout:5568 writesame 0x73bd14d thru 0x73c9a82 data_size 0x33(0xc936 bytes) 2024-09-16T11:15:21.220 INFO:teuthology.orchestra.run.smithi162.stdout:5570 read 0x14b8c26 thru 0x14bb54b (0x2926 bytes) 2024-09-16T11:15:21.220 INFO:teuthology.orchestra.run.smithi162.stdout:5571 write 0x55d71e0 thru 0x55d837b (0x119c bytes) 2024-09-16T11:15:21.227 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.227 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.227 INFO:teuthology.orchestra.run.smithi162.stdout:5572 writesame 0x4436fcb thru 0x44427a8 data_size 0x2d(0xb7de bytes) 2024-09-16T11:15:21.235 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:21.230+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:21.237 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:21.238 INFO:teuthology.orchestra.run.smithi162.stdout:5574 write 0x3ed330a thru 0x3eddf91 (0xac88 bytes) 2024-09-16T11:15:21.247 INFO:teuthology.orchestra.run.smithi162.stdout:5577 read 0x6073c20 thru 0x60780a2 (0x4483 bytes) 2024-09-16T11:15:21.247 INFO:teuthology.orchestra.run.smithi162.stdout:5578 read 0x6f61a07 thru 0x6f6ccda (0xb2d4 bytes) 2024-09-16T11:15:21.247 INFO:teuthology.orchestra.run.smithi162.stdout:5579 read 0x39827da thru 0x398ce78 (0xa69f bytes) 2024-09-16T11:15:21.247 INFO:teuthology.orchestra.run.smithi162.stdout:5581 read 0x4138974 thru 0x413bded (0x347a bytes) 2024-09-16T11:15:21.269 INFO:teuthology.orchestra.run.smithi162.stdout:5582 write 0xe1f19f3 thru 0xe1fbe9f (0xa4ad bytes) 2024-09-16T11:15:21.279 INFO:teuthology.orchestra.run.smithi162.stdout:5583 read 0xde84d03 thru 0xde92c58 (0xdf56 bytes) 2024-09-16T11:15:21.279 INFO:teuthology.orchestra.run.smithi162.stdout:5584 write 0x5684fc1 thru 0x5687e41 (0x2e81 bytes) 2024-09-16T11:15:21.285 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.285 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.285 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.285 INFO:teuthology.orchestra.run.smithi162.stdout:5590 punch from 0x6b8f758 to 0x6b97eb5, (0x875d bytes) 2024-09-16T11:15:21.285 INFO:teuthology.orchestra.run.smithi162.stdout:5591 write 0x22a62d3 thru 0x22a9426 (0x3154 bytes) 2024-09-16T11:15:21.288 INFO:teuthology.orchestra.run.smithi162.stdout:5592 writesame 0xcd00267 thru 0xcd0fcb6 data_size 0x48(0xfa50 bytes) 2024-09-16T11:15:21.297 INFO:teuthology.orchestra.run.smithi162.stdout:5596 read 0x5ed69e0 thru 0x5ee6437 (0xfa58 bytes) 2024-09-16T11:15:21.297 INFO:teuthology.orchestra.run.smithi162.stdout:5597 write 0x1cfe33f thru 0x1d0a526 (0xc1e8 bytes) 2024-09-16T11:15:21.307 INFO:teuthology.orchestra.run.smithi162.stdout:5599 punch from 0x78c5da1 to 0x78cb518, (0x5777 bytes) 2024-09-16T11:15:21.307 INFO:teuthology.orchestra.run.smithi162.stdout:5602 read 0x37d676e thru 0x37df86e (0x9101 bytes) 2024-09-16T11:15:21.307 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.307 INFO:teuthology.orchestra.run.smithi162.stdout:5604 writesame 0xb18e201 thru 0xb1960b5 data_size 0x1(0x7eb5 bytes) 2024-09-16T11:15:21.335 INFO:teuthology.orchestra.run.smithi162.stdout:5605 trunc from 0xe1fbea0 to 0x83e6645 2024-09-16T11:15:21.336 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:21.330+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:21.356 INFO:teuthology.orchestra.run.smithi162.stdout:5606 write 0x6206ec thru 0x620779 (0x8e bytes) 2024-09-16T11:15:21.358 INFO:teuthology.orchestra.run.smithi162.stdout:5607 write 0x51d37fc thru 0x51dd734 (0x9f39 bytes) 2024-09-16T11:15:21.358 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.358 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.368 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:21.379 INFO:teuthology.orchestra.run.smithi162.stdout:5608 trunc from 0x83e6645 to 0xd5dd676 2024-09-16T11:15:21.388 INFO:teuthology.orchestra.run.smithi162.stdout:5609 write 0x619e23d thru 0x61a3c89 (0x5a4d bytes) 2024-09-16T11:15:21.413 INFO:teuthology.orchestra.run.smithi162.stdout:5610 write 0xe0bed09 thru 0xe0c25b8 (0x38b0 bytes) 2024-09-16T11:15:21.425 INFO:teuthology.orchestra.run.smithi162.stdout:5611 writesame 0x2508392 thru 0x25126ce data_size 0x1d(0xa33d bytes) 2024-09-16T11:15:21.429 INFO:teuthology.orchestra.run.smithi162.stdout:5613 flatten 2024-09-16T11:15:21.437 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:21.430+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:21.440 INFO:teuthology.orchestra.run.smithi162.stdout:5614 read 0x848263e thru 0x848bb73 (0x9536 bytes) 2024-09-16T11:15:21.440 INFO:teuthology.orchestra.run.smithi162.stdout:5616 write 0xaf3289f thru 0xaf39966 (0x70c8 bytes) 2024-09-16T11:15:21.450 INFO:teuthology.orchestra.run.smithi162.stdout:5617 write 0x62dfa79 thru 0x62ee84b (0xedd3 bytes) 2024-09-16T11:15:21.454 INFO:teuthology.orchestra.run.smithi162.stdout:5619 read 0x81ba720 thru 0x81c8ec4 (0xe7a5 bytes) 2024-09-16T11:15:21.472 INFO:teuthology.orchestra.run.smithi162.stdout:5620 trunc from 0xe0c25b9 to 0x342b07c 2024-09-16T11:15:21.486 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.486 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:21.510 INFO:teuthology.orchestra.run.smithi162.stdout:5621 write 0x63e9654 thru 0x63ea771 (0x111e bytes) 2024-09-16T11:15:21.517 INFO:teuthology.orchestra.run.smithi162.stdout:5622 punch from 0x5963b7b to 0x5967d7a, (0x41ff bytes) 2024-09-16T11:15:21.517 INFO:teuthology.orchestra.run.smithi162.stdout:5623 punch from 0x40f7cf8 to 0x41046cb, (0xc9d3 bytes) 2024-09-16T11:15:21.517 INFO:teuthology.orchestra.run.smithi162.stdout:5624 writesame 0x34c0895 thru 0x34c989d data_size 0x33(0x9009 bytes) 2024-09-16T11:15:21.531 INFO:teuthology.orchestra.run.smithi162.stdout:5626 trunc from 0x63ea772 to 0x7e68357 2024-09-16T11:15:21.537 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:21.534+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:21.547 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:15:21.553 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T11:15:21.562 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T11:15:21.562 INFO:teuthology.orchestra.run.smithi162.stdout:5628 write 0x872e47d thru 0x8731fc1 (0x3b45 bytes) 2024-09-16T11:15:21.571 INFO:teuthology.orchestra.run.smithi162.stdout:5629 write 0x6d60ba2 thru 0x6d66e14 (0x6273 bytes) 2024-09-16T11:15:21.580 INFO:teuthology.orchestra.run.smithi162.stdout:5630 write 0x1cee4c8 thru 0x1cee75e (0x297 bytes) 2024-09-16T11:15:21.582 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:21.582 INFO:teuthology.orchestra.run.smithi162.stdout:5632 punch from 0x1c843ca to 0x1c912b3, (0xcee9 bytes) 2024-09-16T11:15:21.587 INFO:teuthology.orchestra.run.smithi162.stdout:5636 trunc from 0x8731fc2 to 0xb3ec3fe 2024-09-16T11:15:21.597 INFO:teuthology.orchestra.run.smithi162.stdout:5637 read 0x6cfd88 thru 0x6de08e (0xe307 bytes) 2024-09-16T11:15:21.619 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.619 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.628 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:21.639 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:21.637+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:21.739 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:21.734+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:21.768 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.769 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.777 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:21.841 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:21.838+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:21.884 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:21.884 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:21.893 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:21.942 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:21.941+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:22.039 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.039 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.043 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:22.041+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:22.048 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:22.144 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:22.141+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:22.147 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.147 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.156 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:22.244 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:22.241+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:22.258 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.259 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.268 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:22.345 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:22.342+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:22.368 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.368 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.377 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:22.446 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:22.445+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:22.474 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.474 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.483 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:22.547 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:22.542+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:15:22.559 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T11:15:22.570 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T11:15:22.593 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.593 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.602 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:22.648 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:22.645+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:22.654 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:22.700 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.700 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.709 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:22.749 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:22.749+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:22.785 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:22.806 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:22.807 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:22.807 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:22.816 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:22.825 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:22.847 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:22.862 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:22.861+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:22.868 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:22.890 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:22.909 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:22.931 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:22.934 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:22.934 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:22.934 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:22.934 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:22.934 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:22.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:22.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:22.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:22.952 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:22.946+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:22.963 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:22.963 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:22.963 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:22.972 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:23.053 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:23.049+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:23.067 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:23.067 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:23.068 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:23.068 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:23.077 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:23.154 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:23.150+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:23.222 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 122.357047618, 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:23.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.228 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.189075969000001, 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.229 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:23.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:23.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.189064407999993, 2024-09-16T11:15:23.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:23.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:23.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.241 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.189051206000002, 2024-09-16T11:15:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.242 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:23.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.246 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.189038765999996, 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:23.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.250 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.189025255000004, 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.251 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:23.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:23.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.18901434, 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.255 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:23.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.259 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188988421999994, 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:23.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:23.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188972968000002, 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.265 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:23.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:23.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:23.269 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188963082000001, 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:23.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:23.273 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188953319000007, 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.274 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.278 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188943141999999, 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:23.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.282 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188926660000007, 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.283 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:23.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188911962999995, 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:23.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:23.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188893519999993, 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.292 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:23.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:23.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188876570999994, 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:23.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:23.301 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188857358000007, 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:23.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188840142000004, 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:23.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188818888, 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:23.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:23.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.188798919999996, 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.316 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:23.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.320 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:23.322 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:23.322 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:23.318+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:23.423 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:23.418+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:23.427 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:23.427 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:23.427 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:23.437 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:23.523 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:23.521+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:23.535 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:23.535 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:23.535 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:23.544 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:23.567 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T11:15:23.578 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T11:15:23.624 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:23.618+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:23.645 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.448256158999996, 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.646 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:15:23.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:15:23.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.444386527999995, 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:23.650 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.654 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.444370461000005, 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:23.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:23.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.444354066000002, 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:23.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:23.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:23.662 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.444344491999999, 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.663 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.666 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.444334956000006, 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.667 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:23.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:23.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.746498072999998, 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:23.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:23.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 52.060662819000001, 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:23.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:23.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:23.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.126143475999999, 2024-09-16T11:15:23.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:23.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:23.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.126062805, 2024-09-16T11:15:23.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:23.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.691 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.125941413, 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.696 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.125889219000001, 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.697 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:23.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:23.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:23.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.125832486, 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.703 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:23.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.125785286999999, 2024-09-16T11:15:23.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:23.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:23.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.125735317, 2024-09-16T11:15:23.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:23.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:23.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.125668203, 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:23.718 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:23.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:23.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.123693116999998, 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:23.723 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.123644632000001, 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.728 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.733 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.123527604, 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:23.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:23.738 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.123436429000002, 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:23.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:23.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:23.743 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:23.745 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:23.745 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:23.741+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:23.845 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:23.842+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:23.895 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:23.895 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:23.895 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:23.904 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:23.947 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:23.942+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:24.007 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:24.008 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:24.008 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:24.008 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:24.008 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:24.018 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:24.062 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:24.061+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:24.120 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 154.340552043, 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:24.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:24.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:24.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95262634299999, 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:24.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.952475013, 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:24.134 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:24.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:24.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.141 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.142 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:24.142 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95242352700001, 2024-09-16T11:15:24.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:24.142 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.142 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.142 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.143 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:24.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:24.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95235337899999, 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.149 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:24.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:24.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:24.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95218090899999, 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.155 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:24.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95212258699999, 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.161 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:24.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:24.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.951970467, 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.167 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:24.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.172 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95179194799999, 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.173 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:24.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.95175897499999, 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:24.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:24.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.184 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.39957125000001, 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:24.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.190 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.39944245300001, 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:24.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.196 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.39938410799999, 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:24.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.39932971900001, 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.202 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:24.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:24.207 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 153.39919541200001, 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.208 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:24.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:24.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.61804004300001, 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012150674, 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 92 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498532+0000", 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498533+0000", 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2470000000000001e-06 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498545+0000", 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2043e-05 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498590+0000", 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4412000000000001e-05 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498601+0000", 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1066999999999999e-05 2024-09-16T11:15:24.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505294+0000", 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0066938429999999997 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505478+0000", 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000183213 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.505758+0000", 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00028073099999999999 2024-09-16T11:15:24.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510472+0000", 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0047133499999999998 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510547+0000", 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5308000000000005e-05 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510612+0000", 2024-09-16T11:15:24.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5080999999999996e-05 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510683+0000", 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0378999999999998e-05 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.617952731, 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:24.219 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:24.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:24.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:24.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.617714603, 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:24.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:24.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:24.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:24.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:71747 4.8 e67/31 4:1540c92e:::rbd_data.11b5791b112d.0000000000000009:head v 67'196, mlcod=66'194)", 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 81.937334594000006, 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012056252999999999, 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179237+0000", 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.179238+0000", 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.068e-06 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181554+0000", 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0023154289999999999 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181574+0000", 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0556000000000001e-05 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181584+0000", 2024-09-16T11:15:24.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7159999999999998e-06 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181650+0000", 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5953999999999999e-05 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.181688+0000", 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8118000000000003e-05 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191226+0000", 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0095379690000000003 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:02.191294+0000", 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7442999999999998e-05 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:24.235 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.229142158000002, 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:24.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:24.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.240 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:24.240 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:24.240 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:24.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:24.242 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:24.238+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:24.342 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:24.338+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:24.344 INFO:teuthology.orchestra.run.smithi162.stdout:5638 read 0xababa92 thru 0xabb5945 (0x9eb4 bytes) 2024-09-16T11:15:24.344 INFO:teuthology.orchestra.run.smithi162.stdout:5639 write 0x255d56c thru 0x25613b2 (0x3e47 bytes) 2024-09-16T11:15:24.347 INFO:teuthology.orchestra.run.smithi162.stdout:5640 write 0x74e862d thru 0x74e8dc2 (0x796 bytes) 2024-09-16T11:15:24.357 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:24.358 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:24.358 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:24.358 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:24.360 INFO:teuthology.orchestra.run.smithi162.stdout:5641 trunc from 0xb3ec3fe to 0x7ddb909 2024-09-16T11:15:24.370 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:24.382 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.383 INFO:teuthology.orchestra.run.smithi162.stdout:5645 punch from 0x770c05 to 0x7782b7, (0x76b2 bytes) 2024-09-16T11:15:24.405 INFO:teuthology.orchestra.run.smithi162.stdout:5646 write 0xe746ba2 thru 0xe750374 (0x97d3 bytes) 2024-09-16T11:15:24.421 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.421 INFO:teuthology.orchestra.run.smithi162.stdout:5649 read 0x514d61e thru 0x5157287 (0x9c6a bytes) 2024-09-16T11:15:24.421 INFO:teuthology.orchestra.run.smithi162.stdout:5650 punch from 0xd79dda9 to 0xd79e82c, (0xa83 bytes) 2024-09-16T11:15:24.421 INFO:teuthology.orchestra.run.smithi162.stdout:5651 write 0x886a6c1 thru 0x88714e8 (0x6e28 bytes) 2024-09-16T11:15:24.429 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.429 INFO:teuthology.orchestra.run.smithi162.stdout:5654 read 0x6e766e6 thru 0x6e7ee7d (0x8798 bytes) 2024-09-16T11:15:24.430 INFO:teuthology.orchestra.run.smithi162.stdout:5656 write 0xd0f03c9 thru 0xd0f7fb6 (0x7bee bytes) 2024-09-16T11:15:24.438 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.438 INFO:teuthology.orchestra.run.smithi162.stdout:5658 write 0xd087d3c thru 0xd094881 (0xcb46 bytes) 2024-09-16T11:15:24.441 INFO:teuthology.orchestra.run.smithi162.stdout:5659 writesame 0x671da67 thru 0x67293e1 data_size 0x1(0xb97b bytes) 2024-09-16T11:15:24.443 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:24.438+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:24.475 INFO:teuthology.orchestra.run.smithi162.stdout:5661 read 0xd70ca35 thru 0xd719ac0 (0xd08c bytes) 2024-09-16T11:15:24.475 INFO:teuthology.orchestra.run.smithi162.stdout:5663 write 0x1995e21 thru 0x199ad2c (0x4f0c bytes) 2024-09-16T11:15:24.482 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:24.482 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:24.486 INFO:teuthology.orchestra.run.smithi162.stdout:5664 read 0xbe8154e thru 0xbe8d87e (0xc331 bytes) 2024-09-16T11:15:24.486 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.486 INFO:teuthology.orchestra.run.smithi162.stdout:5666 punch from 0xd145a72 to 0xd1492cf, (0x385d bytes) 2024-09-16T11:15:24.487 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.487 INFO:teuthology.orchestra.run.smithi162.stdout:5668 read 0xdfc5fe6 thru 0xdfd540c (0xf427 bytes) 2024-09-16T11:15:24.487 INFO:teuthology.orchestra.run.smithi162.stdout:5669 writesame 0x9913059 thru 0x991e6b7 data_size 0x1(0xb65f bytes) 2024-09-16T11:15:24.492 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:24.507 INFO:teuthology.orchestra.run.smithi162.stdout:5670 read 0xe40ddd8 thru 0xe4198da (0xbb03 bytes) 2024-09-16T11:15:24.507 INFO:teuthology.orchestra.run.smithi162.stdout:5671 write 0x1143c6e thru 0x114f826 (0xbbb9 bytes) 2024-09-16T11:15:24.517 INFO:teuthology.orchestra.run.smithi162.stdout:5673 write 0x5fe21a4 thru 0x5fe652b (0x4388 bytes) 2024-09-16T11:15:24.544 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:24.541+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:24.548 INFO:teuthology.orchestra.run.smithi162.stdout:5674 trunc from 0xe750375 to 0x7cbea07 2024-09-16T11:15:24.572 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:15:24.576 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T11:15:24.577 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.577 INFO:teuthology.orchestra.run.smithi162.stdout:5676 punch from 0x3a1655e to 0x3a1e284, (0x7d26 bytes) 2024-09-16T11:15:24.577 INFO:teuthology.orchestra.run.smithi162.stdout:5677 write 0x276e95 thru 0x27c8a7 (0x5a13 bytes) 2024-09-16T11:15:24.592 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T11:15:24.604 INFO:teuthology.orchestra.run.smithi162.stdout:5678 write 0xbadac66 thru 0xbadc996 (0x1d31 bytes) 2024-09-16T11:15:24.616 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:24.616 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.43873544100001, 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.617 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:24.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.622 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.436272167, 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:24.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.625 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:24.628 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.43506393200001, 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:24.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.632 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.43299863999999, 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.633 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.432984784, 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:24.637 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:24.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:24.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.43297090900001, 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.645 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.43295714200001, 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.646 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.650 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.43290963299999, 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:24.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.654 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.432623815, 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:24.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.339959999, 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:24.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:24.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.339810817, 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:24.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:24.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351409597, 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:24.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351362637000001, 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351343945, 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.679 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351327745999999, 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:24.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.687 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351312704000001, 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:24.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:24.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:24.692 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351296953999999, 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:24.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351281663999998, 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:24.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351266085999999, 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.701 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:24.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 20.351251107, 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:24.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:24.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:24.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:24.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:24.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:24.710 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:24.712 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:24.713 INFO:teuthology.orchestra.run.smithi162.stdout:5679 trunc from 0xbadc997 to 0xaa1135 2024-09-16T11:15:24.713 INFO:teuthology.orchestra.run.smithi162.stdout:5680 read 0x52e1b9 thru 0x53158b (0x33d3 bytes) 2024-09-16T11:15:24.713 INFO:teuthology.orchestra.run.smithi162.stdout:5681 write 0x52f5802 thru 0x52f8dfa (0x35f9 bytes) 2024-09-16T11:15:24.713 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:24.713 INFO:teuthology.orchestra.run.smithi162.stdout:5684 trunc from 0x52f8dfb to 0xbe374df 2024-09-16T11:15:24.713 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:24.706+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:24.714 INFO:teuthology.orchestra.run.smithi162.stdout:5685 read 0x47ca6c7 thru 0x47d818c (0xdac6 bytes) 2024-09-16T11:15:24.714 INFO:teuthology.orchestra.run.smithi162.stdout:5686 read 0x61898cb thru 0x61944b3 (0xabe9 bytes) 2024-09-16T11:15:24.714 INFO:teuthology.orchestra.run.smithi162.stdout:5687 punch from 0x1cb0510 to 0x1cbaf0a, (0xa9fa bytes) 2024-09-16T11:15:24.715 INFO:teuthology.orchestra.run.smithi162.stdout:5689 write 0x6c8f34d thru 0x6c91f07 (0x2bbb bytes) 2024-09-16T11:15:24.737 INFO:teuthology.orchestra.run.smithi162.stdout:5690 trunc from 0xbe374df to 0x4b9e68b 2024-09-16T11:15:24.787 INFO:teuthology.orchestra.run.smithi162.stdout:5691 write 0xe9df62e thru 0xe9e93aa (0x9d7d bytes) 2024-09-16T11:15:24.798 INFO:teuthology.orchestra.run.smithi162.stdout:5692 writesame 0x90b28a0 thru 0x90c0603 data_size 0x6(0xdd64 bytes) 2024-09-16T11:15:24.813 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:24.806+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:24.834 INFO:teuthology.orchestra.run.smithi162.stdout:5693 trunc from 0xe9e93ab to 0x16637d 2024-09-16T11:15:24.860 INFO:teuthology.orchestra.run.smithi162.stdout:5694 read 0x153f11 thru 0x1584d7 (0x45c7 bytes) 2024-09-16T11:15:24.864 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:24.864 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:24.875 INFO:teuthology.orchestra.run.smithi162.stdout:5695 writesame 0x4190a51 thru 0x419db25 data_size 0x1(0xd0d5 bytes) 2024-09-16T11:15:24.879 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:24.887 INFO:teuthology.orchestra.run.smithi162.stdout:5697 write 0x52e9653 thru 0x52f7be3 (0xe591 bytes) 2024-09-16T11:15:24.913 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:24.913+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:24.921 INFO:teuthology.orchestra.run.smithi162.stdout:5698 writesame 0xc04cb4e thru 0xc05a9bb data_size 0x47(0xde6e bytes) 2024-09-16T11:15:24.921 INFO:teuthology.orchestra.run.smithi162.stdout:5700 read 0x4385ea0 thru 0x438bcf2 (0x5e53 bytes) 2024-09-16T11:15:24.921 INFO:teuthology.orchestra.run.smithi162.stdout:5701 read 0xa50f733 thru 0xa517582 (0x7e50 bytes) 2024-09-16T11:15:24.921 INFO:teuthology.orchestra.run.smithi162.stdout:5702 write 0x5b78904 thru 0x5b8083d (0x7f3a bytes) 2024-09-16T11:15:24.936 INFO:teuthology.orchestra.run.smithi162.stdout:5703 read 0x9322188 thru 0x9323911 (0x178a bytes) 2024-09-16T11:15:24.936 INFO:teuthology.orchestra.run.smithi162.stdout:5704 write 0x5c1f09 thru 0x5c86ca (0x67c2 bytes) 2024-09-16T11:15:24.944 INFO:teuthology.orchestra.run.smithi162.stdout:5706 writesame 0xa0ff5a1 thru 0xa10ab37 data_size 0x1d(0xb597 bytes) 2024-09-16T11:15:24.945 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:24.953 INFO:teuthology.orchestra.run.smithi162.stdout:5708 write 0x65e4efd thru 0x65ef81a (0xa91e bytes) 2024-09-16T11:15:24.962 INFO:teuthology.orchestra.run.smithi162.stdout:5709 punch from 0xbe768b7 to 0xbe7b9ff, (0x5148 bytes) 2024-09-16T11:15:24.962 INFO:teuthology.orchestra.run.smithi162.stdout:5710 read 0x76002ed thru 0x760803e (0x7d52 bytes) 2024-09-16T11:15:24.973 INFO:teuthology.orchestra.run.smithi162.stdout:5711 writesame 0xda0ec98 thru 0xda112a0 data_size 0xd(0x2609 bytes) 2024-09-16T11:15:24.986 INFO:teuthology.orchestra.run.smithi162.stdout:5712 trunc from 0xda112a1 to 0x5c350b5 2024-09-16T11:15:25.014 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:25.010+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:25.022 INFO:teuthology.orchestra.run.smithi162.stdout:5713 trunc from 0x5c350b5 to 0xd139864 2024-09-16T11:15:25.022 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:25.022 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:25.023 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:25.030 INFO:teuthology.orchestra.run.smithi162.stdout:5714 writesame 0x59bd560 thru 0x59c1c35 data_size 0x1(0x46d6 bytes) 2024-09-16T11:15:25.037 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:25.047 INFO:teuthology.orchestra.run.smithi162.stdout:5715 write 0x970d3bb thru 0x9716e89 (0x9acf bytes) 2024-09-16T11:15:25.057 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.057 INFO:teuthology.orchestra.run.smithi162.stdout:5721 read 0xcf5fe28 thru 0xcf6614d (0x6326 bytes) 2024-09-16T11:15:25.057 INFO:teuthology.orchestra.run.smithi162.stdout:5723 read 0x6a5d40d thru 0x6a60319 (0x2f0d bytes) 2024-09-16T11:15:25.058 INFO:teuthology.orchestra.run.smithi162.stdout:5724 writesame 0xb54520d thru 0xb549810 data_size 0x2(0x4604 bytes) 2024-09-16T11:15:25.068 INFO:teuthology.orchestra.run.smithi162.stdout:5725 read 0x917327 thru 0x918aac (0x1786 bytes) 2024-09-16T11:15:25.068 INFO:teuthology.orchestra.run.smithi162.stdout:5726 write 0x32eaa71 thru 0x32f3328 (0x88b8 bytes) 2024-09-16T11:15:25.077 INFO:teuthology.orchestra.run.smithi162.stdout:5728 read 0x2072b5e thru 0x2076502 (0x39a5 bytes) 2024-09-16T11:15:25.077 INFO:teuthology.orchestra.run.smithi162.stdout:5729 write 0x43fdf21 thru 0x440d9b2 (0xfa92 bytes) 2024-09-16T11:15:25.086 INFO:teuthology.orchestra.run.smithi162.stdout:5734 writesame 0x4e8784 thru 0x4f6144 data_size 0x1(0xd9c1 bytes) 2024-09-16T11:15:25.105 INFO:teuthology.orchestra.run.smithi162.stdout:5735 punch from 0x595e0ca to 0x595eeee, (0xe24 bytes) 2024-09-16T11:15:25.106 INFO:teuthology.orchestra.run.smithi162.stdout:5737 punch from 0xbe0ded4 to 0xbe1d9ea, (0xfb16 bytes) 2024-09-16T11:15:25.106 INFO:teuthology.orchestra.run.smithi162.stdout:5738 read 0x9e2b9af thru 0x9e368e6 (0xaf38 bytes) 2024-09-16T11:15:25.106 INFO:teuthology.orchestra.run.smithi162.stdout:5741 punch from 0x20b031c to 0x20bea87, (0xe76b bytes) 2024-09-16T11:15:25.106 INFO:teuthology.orchestra.run.smithi162.stdout:5742 write 0xa6a9324 thru 0xa6b6b32 (0xd80f bytes) 2024-09-16T11:15:25.115 INFO:teuthology.orchestra.run.smithi162.stdout:5743 punch from 0x872ebb7 to 0x8737a70, (0x8eb9 bytes) 2024-09-16T11:15:25.115 INFO:teuthology.orchestra.run.smithi162.stdout:5744 read 0x8a3db99 thru 0x8a41879 (0x3ce1 bytes) 2024-09-16T11:15:25.115 INFO:teuthology.orchestra.run.smithi162.stdout:5745 write 0x4a6cbb9 thru 0x4a7a4e4 (0xd92c bytes) 2024-09-16T11:15:25.116 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:25.116 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:25.110+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:25.124 INFO:teuthology.orchestra.run.smithi162.stdout:5746 read 0x87dbc20 thru 0x87e6e2a (0xb20b bytes) 2024-09-16T11:15:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:5747 read 0x8dc2336 thru 0x8dcd0d5 (0xada0 bytes) 2024-09-16T11:15:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:5749 punch from 0x6d96508 to 0x6da44e1, (0xdfd9 bytes) 2024-09-16T11:15:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:5750 read 0x6d3908 thru 0x6d7fc6 (0x46bf bytes) 2024-09-16T11:15:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:5751 read 0xb6117bd thru 0xb620b18 (0xf35c bytes) 2024-09-16T11:15:25.125 INFO:teuthology.orchestra.run.smithi162.stdout:5752 write 0x1d3df45 thru 0x1d45a9d (0x7b59 bytes) 2024-09-16T11:15:25.133 INFO:teuthology.orchestra.run.smithi162.stdout:5754 writesame 0x1dc9f52 thru 0x1dcded7 data_size 0x2f(0x3f86 bytes) 2024-09-16T11:15:25.137 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.138 INFO:teuthology.orchestra.run.smithi162.stdout:5757 trunc from 0xd139864 to 0xe0112d1 2024-09-16T11:15:25.140 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:25.147 INFO:teuthology.orchestra.run.smithi162.stdout:5758 trunc from 0xe0112d1 to 0xe635d68 2024-09-16T11:15:25.155 INFO:teuthology.orchestra.run.smithi162.stdout:5759 read 0x7040992 thru 0x7041e02 (0x1471 bytes) 2024-09-16T11:15:25.156 INFO:teuthology.orchestra.run.smithi162.stdout:5760 read 0x61cd90d thru 0x61d4d2d (0x7421 bytes) 2024-09-16T11:15:25.156 INFO:teuthology.orchestra.run.smithi162.stdout:5762 read 0x43cdee3 thru 0x43d15e1 (0x36ff bytes) 2024-09-16T11:15:25.156 INFO:teuthology.orchestra.run.smithi162.stdout:5763 read 0x302868b thru 0x302ddaf (0x5725 bytes) 2024-09-16T11:15:25.157 INFO:teuthology.orchestra.run.smithi162.stdout:5764 read 0x1e12974 thru 0x1e17252 (0x48df bytes) 2024-09-16T11:15:25.157 INFO:teuthology.orchestra.run.smithi162.stdout:5765 write 0x402c023 thru 0x4033076 (0x7054 bytes) 2024-09-16T11:15:25.164 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:25.167 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.167 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.169 INFO:teuthology.orchestra.run.smithi162.stdout:5766 read 0x552c61b thru 0x5531640 (0x5026 bytes) 2024-09-16T11:15:25.172 INFO:teuthology.orchestra.run.smithi162.stdout:5768 trunc from 0xe635d68 to 0xd139c74 2024-09-16T11:15:25.180 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:25.181 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.181 INFO:teuthology.orchestra.run.smithi162.stdout:5771 write 0xb2cea8d thru 0xb2d5199 (0x670d bytes) 2024-09-16T11:15:25.185 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:25.193 INFO:teuthology.orchestra.run.smithi162.stdout:5773 read 0x8e6c36 thru 0x8f20d0 (0xb49b bytes) 2024-09-16T11:15:25.194 INFO:teuthology.orchestra.run.smithi162.stdout:5775 punch from 0x716ef60 to 0x717467f, (0x571f bytes) 2024-09-16T11:15:25.194 INFO:teuthology.orchestra.run.smithi162.stdout:5776 write 0x2a38bab thru 0x2a45274 (0xc6ca bytes) 2024-09-16T11:15:25.204 INFO:teuthology.orchestra.run.smithi162.stdout:5777 read 0x661e4ad thru 0x662a3b9 (0xbf0d bytes) 2024-09-16T11:15:25.205 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.205 INFO:teuthology.orchestra.run.smithi162.stdout:5780 read 0xca38798 thru 0xca47e13 (0xf67c bytes) 2024-09-16T11:15:25.205 INFO:teuthology.orchestra.run.smithi162.stdout:5781 write 0xa38e01b thru 0xa396db0 (0x8d96 bytes) 2024-09-16T11:15:25.214 INFO:teuthology.orchestra.run.smithi162.stdout:5782 read 0x4c96c70 thru 0x4c9ad2d (0x40be bytes) 2024-09-16T11:15:25.214 INFO:teuthology.orchestra.run.smithi162.stdout:5784 writesame 0xc43d9de thru 0xc440ab0 data_size 0x1(0x30d3 bytes) 2024-09-16T11:15:25.216 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:25.210+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:25.220 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:25.224 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:5787 read 0x4c6963 thru 0x4c741b (0xab9 bytes) 2024-09-16T11:15:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:5788 read 0xa0f0c9b thru 0xa0ff153 (0xe4b9 bytes) 2024-09-16T11:15:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:5790 punch from 0x5b35284 to 0x5b377be, (0x253a bytes) 2024-09-16T11:15:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:5792 punch from 0x57958d8 to 0x57a0c2a, (0xb352 bytes) 2024-09-16T11:15:25.226 INFO:teuthology.orchestra.run.smithi162.stdout:5793 read 0x2f06d67 thru 0x2f0c9e4 (0x5c7e bytes) 2024-09-16T11:15:25.226 INFO:teuthology.orchestra.run.smithi162.stdout:5794 writesame 0xbab4ff3 thru 0xbab7729 data_size 0x1(0x2737 bytes) 2024-09-16T11:15:25.237 INFO:teuthology.orchestra.run.smithi162.stdout:5795 writesame 0x86bdce5 thru 0x86c143e data_size 0x1a(0x375a bytes) 2024-09-16T11:15:25.246 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:25.249 INFO:teuthology.orchestra.run.smithi162.stdout:5796 writesame 0x9cfde82 thru 0x9cfe8ad data_size 0x7(0xa2c bytes) 2024-09-16T11:15:25.266 INFO:teuthology.orchestra.run.smithi162.stdout:5797 trunc from 0xd139c74 to 0x635b2fa 2024-09-16T11:15:25.271 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:25.299 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:25.301 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.302 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.303 INFO:teuthology.orchestra.run.smithi162.stdout:5798 write 0x6b89a44 thru 0x6b93f06 (0xa4c3 bytes) 2024-09-16T11:15:25.313 INFO:teuthology.orchestra.run.smithi162.stdout:5799 write 0x2333122 thru 0x233a01e (0x6efd bytes) 2024-09-16T11:15:25.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:25.324 INFO:teuthology.orchestra.run.smithi162.stdout:5800 read 0xf07c00 thru 0xf10f4d (0x934e bytes) 2024-09-16T11:15:25.324 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.325 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:25.318+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:25.334 INFO:teuthology.orchestra.run.smithi162.stdout:5804 trunc from 0x6b93f07 to 0xca2c780 2024-09-16T11:15:25.353 INFO:teuthology.orchestra.run.smithi162.stdout:5806 write 0xd29f2ae thru 0xd2a9ebf (0xac12 bytes) 2024-09-16T11:15:25.364 INFO:teuthology.orchestra.run.smithi162.stdout:5807 punch from 0x2dffca4 to 0x2e03a5b, (0x3db7 bytes) 2024-09-16T11:15:25.372 INFO:teuthology.orchestra.run.smithi162.stdout:5812 trunc from 0xd2a9ec0 to 0x7cec9b4 2024-09-16T11:15:25.390 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.400 INFO:teuthology.orchestra.run.smithi162.stdout:5814 trunc from 0x7cec9b4 to 0xe26b373 2024-09-16T11:15:25.409 INFO:teuthology.orchestra.run.smithi162.stdout:5815 punch from 0x6139e12 to 0x613bf9d, (0x218b bytes) 2024-09-16T11:15:25.409 INFO:teuthology.orchestra.run.smithi162.stdout:5816 read 0x6ebfa7a thru 0x6ec5334 (0x58bb bytes) 2024-09-16T11:15:25.409 INFO:teuthology.orchestra.run.smithi162.stdout:5817 writesame 0x3a6e401 thru 0x3a6fc02 data_size 0x7(0x1802 bytes) 2024-09-16T11:15:25.418 INFO:teuthology.orchestra.run.smithi162.stdout:5819 punch from 0xd5b28c0 to 0xd5c0877, (0xdfb7 bytes) 2024-09-16T11:15:25.418 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.418 INFO:teuthology.orchestra.run.smithi162.stdout:5822 punch from 0x324f8cf to 0x325af5f, (0xb690 bytes) 2024-09-16T11:15:25.418 INFO:teuthology.orchestra.run.smithi162.stdout:5823 punch from 0x2ac5932 to 0x2ac9017, (0x36e5 bytes) 2024-09-16T11:15:25.419 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.419 INFO:teuthology.orchestra.run.smithi162.stdout:5825 write 0x7c6caae thru 0x7c70101 (0x3654 bytes) 2024-09-16T11:15:25.420 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:25.417+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:25.431 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:25.431 INFO:teuthology.orchestra.run.smithi162.stdout:5827 punch from 0xa8468c7 to 0xa847684, (0xdbd bytes) 2024-09-16T11:15:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.441 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.448 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T11:15:25.449 INFO:teuthology.orchestra.run.smithi162.stdout:5828 trunc from 0xe26b373 to 0x4d648f7 2024-09-16T11:15:25.451 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:15:25.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:25.466 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T11:15:25.519 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:25.514+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:25.617 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.617 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.619 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:25.617+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:25.628 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:25.721 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:25.717+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:25.734 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.734 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.744 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:25.822 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:25.821+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:25.835 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.835 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.845 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:25.923 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:25.918+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:25.940 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:25.940 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:25.949 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:26.024 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:26.018+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:26.057 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.057 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.067 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:26.125 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:26.121+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:26.163 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.163 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.172 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:26.226 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:26.222+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:26.268 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.268 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.277 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:26.327 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:26.325+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:26.378 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.378 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.388 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:26.427 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:26.422+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:26.452 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T11:15:26.469 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T11:15:26.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:15:26.492 INFO:teuthology.orchestra.run.smithi162.stdout:5830 writesame 0xc61f87a thru 0xc62255e data_size 0x3(0x2ce5 bytes) 2024-09-16T11:15:26.492 INFO:teuthology.orchestra.run.smithi162.stdout:5831 punch from 0xbde7a63 to 0xbdf076c, (0x8d09 bytes) 2024-09-16T11:15:26.497 INFO:teuthology.orchestra.run.smithi162.stdout:5833 trunc from 0xc62255f to 0x99fc96d 2024-09-16T11:15:26.508 INFO:teuthology.orchestra.run.smithi162.stdout:5834 punch from 0x34e550d to 0x34edb1b, (0x860e bytes) 2024-09-16T11:15:26.508 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:26.508 INFO:teuthology.orchestra.run.smithi162.stdout:5837 punch from 0x5cda510 to 0x5ce7228, (0xcd18 bytes) 2024-09-16T11:15:26.508 INFO:teuthology.orchestra.run.smithi162.stdout:5838 write 0x990d912 thru 0x9910489 (0x2b78 bytes) 2024-09-16T11:15:26.519 INFO:teuthology.orchestra.run.smithi162.stdout:5839 punch from 0x666a6d6 to 0x666e490, (0x3dba bytes) 2024-09-16T11:15:26.519 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:26.528 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:26.525+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:26.533 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.533 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.539 INFO:teuthology.orchestra.run.smithi162.stdout:5841 write 0xe6a2744 thru 0xe6b2525 (0xfde2 bytes) 2024-09-16T11:15:26.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:26.549 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:26.549 INFO:teuthology.orchestra.run.smithi162.stdout:5843 writesame 0xbc7e1ab thru 0xbc87fcf data_size 0x5(0x9e25 bytes) 2024-09-16T11:15:26.561 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:26.561 INFO:teuthology.orchestra.run.smithi162.stdout:5847 writesame 0xcea48bb thru 0xceaa397 data_size 0x1(0x5add bytes) 2024-09-16T11:15:26.578 INFO:teuthology.orchestra.run.smithi162.stdout:5849 read 0x595e87 thru 0x5a2a26 (0xcba0 bytes) 2024-09-16T11:15:26.579 INFO:teuthology.orchestra.run.smithi162.stdout:5850 writesame 0x9131ea4 thru 0x9137d51 data_size 0x1(0x5eae bytes) 2024-09-16T11:15:26.629 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:26.626+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:26.643 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.643 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.653 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:26.730 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:26.726+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:26.757 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.757 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.766 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:26.786 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:26.789 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:15:26.802 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T11:15:26.434858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1735592714,"num_objects":2744,"num_object_clones":554,"num_object_copies":5488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2741,"num_whiteouts":712,"num_read":239365,"num_read_kb":1408823,"num_write":24628,"num_write_kb":748635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4158,"num_bytes_recovered":483874757,"num_keys_recovered":573,"num_objects_omap":44,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18478,"ondisk_log_size":18478,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":3131884,"kb_used_data":1004604,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":657470996,"statfs":{"total":676457349120,"available":673250299904,"internally_reserved":0,"allocated":1028714496,"data_stored":1109010460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.889394"},"pg_stats":[{"pgid":"4.1f","version":"134'354","reported_seq":5881,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479509+0000","last_change":"2024-09-16T11:14:58.670830+0000","last_active":"2024-09-16T11:15:21.479509+0000","last_peered":"2024-09-16T11:15:21.479509+0000","last_clean":"2024-09-16T11:15:21.479509+0000","last_became_active":"2024-09-16T11:14:58.669893+0000","last_became_peered":"2024-09-16T11:14:58.669893+0000","last_unstale":"2024-09-16T11:15:21.479509+0000","last_undegraded":"2024-09-16T11:15:21.479509+0000","last_fullsized":"2024-09-16T11:15:21.479509+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:03.555919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002295123,"stat_sum":{"num_bytes":32245286,"num_objects":93,"num_object_clones":23,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":26,"num_read":6299,"num_read_kb":31192,"num_write":332,"num_write_kb":11309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8822592,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"135'669","reported_seq":10535,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.539202+0000","last_change":"2024-09-16T11:13:15.329907+0000","last_active":"2024-09-16T11:15:21.539202+0000","last_peered":"2024-09-16T11:15:21.539202+0000","last_clean":"2024-09-16T11:15:21.539202+0000","last_became_active":"2024-09-16T11:13:13.129077+0000","last_became_peered":"2024-09-16T11:13:13.129077+0000","last_unstale":"2024-09-16T11:15:21.539202+0000","last_undegraded":"2024-09-16T11:15:21.539202+0000","last_fullsized":"2024-09-16T11:15:21.539202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:56.981990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050881799999999997,"stat_sum":{"num_bytes":67527044,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":21,"num_read":12400,"num_read_kb":91410,"num_write":861,"num_write_kb":42836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"134'408","reported_seq":4438,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.427703+0000","last_change":"2024-09-16T11:13:15.119814+0000","last_active":"2024-09-16T11:15:21.427703+0000","last_peered":"2024-09-16T11:15:21.427703+0000","last_clean":"2024-09-16T11:15:21.427703+0000","last_became_active":"2024-09-16T11:13:13.110835+0000","last_became_peered":"2024-09-16T11:13:13.110835+0000","last_unstale":"2024-09-16T11:15:21.427703+0000","last_undegraded":"2024-09-16T11:15:21.427703+0000","last_fullsized":"2024-09-16T11:15:21.427703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:25.687407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063997100000000003,"stat_sum":{"num_bytes":46346359,"num_objects":83,"num_object_clones":18,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":19,"num_read":4677,"num_read_kb":26057,"num_write":540,"num_write_kb":14465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"134'884","reported_seq":8133,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.305659+0000","last_change":"2024-09-16T11:14:24.176887+0000","last_active":"2024-09-16T11:15:21.305659+0000","last_peered":"2024-09-16T11:15:21.305659+0000","last_clean":"2024-09-16T11:15:21.305659+0000","last_became_active":"2024-09-16T11:14:24.105273+0000","last_became_peered":"2024-09-16T11:14:24.105273+0000","last_unstale":"2024-09-16T11:15:21.305659+0000","last_undegraded":"2024-09-16T11:15:21.305659+0000","last_fullsized":"2024-09-16T11:15:21.305659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:10.103253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038046360000000001,"stat_sum":{"num_bytes":40917279,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":26,"num_read":10975,"num_read_kb":37510,"num_write":1275,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2431957,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"134'420","reported_seq":7136,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.486025+0000","last_change":"2024-09-16T11:15:07.454596+0000","last_active":"2024-09-16T11:15:21.486025+0000","last_peered":"2024-09-16T11:15:21.486025+0000","last_clean":"2024-09-16T11:15:21.486025+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:15:21.486025+0000","last_undegraded":"2024-09-16T11:15:21.486025+0000","last_fullsized":"2024-09-16T11:15:21.486025+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029419009999999998,"stat_sum":{"num_bytes":20659487,"num_objects":76,"num_object_clones":12,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":18,"num_read":7224,"num_read_kb":24218,"num_write":558,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"134'556","reported_seq":7250,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.560894+0000","last_change":"2024-09-16T11:15:05.439505+0000","last_active":"2024-09-16T11:15:21.560894+0000","last_peered":"2024-09-16T11:15:21.560894+0000","last_clean":"2024-09-16T11:15:21.560894+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:15:21.560894+0000","last_undegraded":"2024-09-16T11:15:21.560894+0000","last_fullsized":"2024-09-16T11:15:21.560894+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024433810000000001,"stat_sum":{"num_bytes":45458930,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":18,"num_read":7891,"num_read_kb":40418,"num_write":734,"num_write_kb":21939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"134'663","reported_seq":2719,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.196724+0000","last_change":"2024-09-16T11:14:23.090655+0000","last_active":"2024-09-16T11:15:21.196724+0000","last_peered":"2024-09-16T11:15:21.196724+0000","last_clean":"2024-09-16T11:15:21.196724+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:15:21.196724+0000","last_undegraded":"2024-09-16T11:15:21.196724+0000","last_fullsized":"2024-09-16T11:15:21.196724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072097299999999997,"stat_sum":{"num_bytes":54271658,"num_objects":99,"num_object_clones":23,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":29,"num_read":4451,"num_read_kb":20695,"num_write":1023,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"134'609","reported_seq":4830,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541174+0000","last_change":"2024-09-16T11:15:08.474104+0000","last_active":"2024-09-16T11:15:21.541174+0000","last_peered":"2024-09-16T11:15:21.541174+0000","last_clean":"2024-09-16T11:15:21.541174+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:15:21.541174+0000","last_undegraded":"2024-09-16T11:15:21.541174+0000","last_fullsized":"2024-09-16T11:15:21.541174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055416799999999996,"stat_sum":{"num_bytes":84203863,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":17,"num_read":6285,"num_read_kb":35627,"num_write":701,"num_write_kb":29109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"134'1467","reported_seq":6095,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.526860+0000","last_change":"2024-09-16T11:13:15.120697+0000","last_active":"2024-09-16T11:15:21.526860+0000","last_peered":"2024-09-16T11:15:21.526860+0000","last_clean":"2024-09-16T11:15:21.526860+0000","last_became_active":"2024-09-16T11:13:13.131852+0000","last_became_peered":"2024-09-16T11:13:13.131852+0000","last_unstale":"2024-09-16T11:15:21.526860+0000","last_undegraded":"2024-09-16T11:15:21.526860+0000","last_fullsized":"2024-09-16T11:15:21.526860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1467,"log_dups_size":0,"ondisk_log_size":1467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:23.813033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041000900000000002,"stat_sum":{"num_bytes":26038056,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":13,"num_read":11635,"num_read_kb":16377,"num_write":2567,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"134'372","reported_seq":7041,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.192242+0000","last_change":"2024-09-16T11:13:15.330097+0000","last_active":"2024-09-16T11:15:21.192242+0000","last_peered":"2024-09-16T11:15:21.192242+0000","last_clean":"2024-09-16T11:15:21.192242+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:15:21.192242+0000","last_undegraded":"2024-09-16T11:15:21.192242+0000","last_fullsized":"2024-09-16T11:15:21.192242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061357700000000002,"stat_sum":{"num_bytes":59976669,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":22,"num_read":7444,"num_read_kb":49605,"num_write":444,"num_write_kb":16154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"134'368","reported_seq":3465,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479999+0000","last_change":"2024-09-16T11:14:58.670333+0000","last_active":"2024-09-16T11:15:21.479999+0000","last_peered":"2024-09-16T11:15:21.479999+0000","last_clean":"2024-09-16T11:15:21.479999+0000","last_became_active":"2024-09-16T11:14:58.668312+0000","last_became_peered":"2024-09-16T11:14:58.668312+0000","last_unstale":"2024-09-16T11:15:21.479999+0000","last_undegraded":"2024-09-16T11:15:21.479999+0000","last_fullsized":"2024-09-16T11:15:21.479999+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:09:03.441602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065955699999999996,"stat_sum":{"num_bytes":47454015,"num_objects":86,"num_object_clones":18,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":30,"num_read":3239,"num_read_kb":16381,"num_write":343,"num_write_kb":26293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":9395442,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"134'533","reported_seq":5932,"reported_epoch":135,"state":"peering","last_fresh":"2024-09-16T11:15:21.547318+0000","last_change":"2024-09-16T11:15:21.547592+0000","last_active":"2024-09-16T11:15:21.547297+0000","last_peered":"2024-09-16T11:15:21.356909+0000","last_clean":"2024-09-16T11:15:21.356909+0000","last_became_active":"2024-09-16T11:15:17.513082+0000","last_became_peered":"2024-09-16T11:15:17.513082+0000","last_unstale":"2024-09-16T11:15:21.547318+0000","last_undegraded":"2024-09-16T11:15:21.547318+0000","last_fullsized":"2024-09-16T11:15:21.547318+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:43:51.413401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066315600000000003,"stat_sum":{"num_bytes":82278726,"num_objects":87,"num_object_clones":22,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":26,"num_read":6953,"num_read_kb":59576,"num_write":511,"num_write_kb":27647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":8536088,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"135'440","reported_seq":3953,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541635+0000","last_change":"2024-09-16T11:15:05.443267+0000","last_active":"2024-09-16T11:15:21.541635+0000","last_peered":"2024-09-16T11:15:21.541635+0000","last_clean":"2024-09-16T11:15:21.541635+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:15:21.541635+0000","last_undegraded":"2024-09-16T11:15:21.541635+0000","last_fullsized":"2024-09-16T11:15:21.541635+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028235669999999999,"stat_sum":{"num_bytes":46436955,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":16,"num_read":4462,"num_read_kb":38147,"num_write":479,"num_write_kb":23350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"127'357","reported_seq":4747,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T11:15:16.506804+0000","last_change":"2024-09-16T11:15:11.447651+0000","last_active":"2024-09-16T11:15:16.506804+0000","last_peered":"2024-09-16T11:15:16.506804+0000","last_clean":"2024-09-16T11:15:16.506804+0000","last_became_active":"2024-09-16T11:15:11.445111+0000","last_became_peered":"2024-09-16T11:15:11.445111+0000","last_unstale":"2024-09-16T11:15:16.506804+0000","last_undegraded":"2024-09-16T11:15:16.506804+0000","last_fullsized":"2024-09-16T11:15:16.506804+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:42.682703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145994,"stat_sum":{"num_bytes":84902425,"num_objects":85,"num_object_clones":18,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":19,"num_read":4502,"num_read_kb":52887,"num_write":603,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":15771908,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"134'550","reported_seq":6717,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479323+0000","last_change":"2024-09-16T11:13:15.119821+0000","last_active":"2024-09-16T11:15:21.479323+0000","last_peered":"2024-09-16T11:15:21.479323+0000","last_clean":"2024-09-16T11:15:21.479323+0000","last_became_active":"2024-09-16T11:13:13.131936+0000","last_became_peered":"2024-09-16T11:13:13.131936+0000","last_unstale":"2024-09-16T11:15:21.479323+0000","last_undegraded":"2024-09-16T11:15:21.479323+0000","last_fullsized":"2024-09-16T11:15:21.479323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:24.053000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059135899999999996,"stat_sum":{"num_bytes":101583801,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":7906,"num_read_kb":96144,"num_write":620,"num_write_kb":44786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"135'498","reported_seq":6630,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.546518+0000","last_change":"2024-09-16T11:14:45.570341+0000","last_active":"2024-09-16T11:15:21.546518+0000","last_peered":"2024-09-16T11:15:21.546518+0000","last_clean":"2024-09-16T11:15:21.546518+0000","last_became_active":"2024-09-16T11:14:45.569559+0000","last_became_peered":"2024-09-16T11:14:45.569559+0000","last_unstale":"2024-09-16T11:15:21.546518+0000","last_undegraded":"2024-09-16T11:15:21.546518+0000","last_fullsized":"2024-09-16T11:15:21.546518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:51.205615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050522900000000005,"stat_sum":{"num_bytes":50584171,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":24,"num_read":7334,"num_read_kb":37445,"num_write":682,"num_write_kb":22939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":18357825,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570472+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:15:20.570472+0000","last_peered":"2024-09-16T11:15:20.570472+0000","last_clean":"2024-09-16T11:15:20.570472+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:15:20.570472+0000","last_undegraded":"2024-09-16T11:15:20.570472+0000","last_fullsized":"2024-09-16T11:15:20.570472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"134'786","reported_seq":4933,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.344084+0000","last_change":"2024-09-16T11:15:08.474134+0000","last_active":"2024-09-16T11:15:21.344084+0000","last_peered":"2024-09-16T11:15:21.344084+0000","last_clean":"2024-09-16T11:15:21.344084+0000","last_became_active":"2024-09-16T11:15:08.472016+0000","last_became_peered":"2024-09-16T11:15:08.472016+0000","last_unstale":"2024-09-16T11:15:21.344084+0000","last_undegraded":"2024-09-16T11:15:21.344084+0000","last_fullsized":"2024-09-16T11:15:21.344084+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:25.422198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071029199999999995,"stat_sum":{"num_bytes":36064192,"num_objects":88,"num_object_clones":17,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":28,"num_read":7145,"num_read_kb":17284,"num_write":995,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":12894750,"num_keys_recovered":93,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":129,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.546596+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:15:20.546596+0000","last_peered":"2024-09-16T11:15:20.546596+0000","last_clean":"2024-09-16T11:15:20.546596+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:15:20.546596+0000","last_undegraded":"2024-09-16T11:15:20.546596+0000","last_fullsized":"2024-09-16T11:15:20.546596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"134'332","reported_seq":4449,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.567960+0000","last_change":"2024-09-16T11:13:15.119575+0000","last_active":"2024-09-16T11:15:20.567960+0000","last_peered":"2024-09-16T11:15:20.567960+0000","last_clean":"2024-09-16T11:15:20.567960+0000","last_became_active":"2024-09-16T11:13:13.131951+0000","last_became_peered":"2024-09-16T11:13:13.131951+0000","last_unstale":"2024-09-16T11:15:20.567960+0000","last_undegraded":"2024-09-16T11:15:20.567960+0000","last_fullsized":"2024-09-16T11:15:20.567960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:26.039221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051524000000000003,"stat_sum":{"num_bytes":48446784,"num_objects":91,"num_object_clones":19,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":28,"num_read":4421,"num_read_kb":35881,"num_write":347,"num_write_kb":11458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571366+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:15:20.571366+0000","last_peered":"2024-09-16T11:15:20.571366+0000","last_clean":"2024-09-16T11:15:20.571366+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:15:20.571366+0000","last_undegraded":"2024-09-16T11:15:20.571366+0000","last_fullsized":"2024-09-16T11:15:20.571366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"134'830","reported_seq":16427,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.483283+0000","last_change":"2024-09-16T11:14:58.670274+0000","last_active":"2024-09-16T11:15:21.483283+0000","last_peered":"2024-09-16T11:15:21.483283+0000","last_clean":"2024-09-16T11:15:21.483283+0000","last_became_active":"2024-09-16T11:14:58.669120+0000","last_became_peered":"2024-09-16T11:14:58.669120+0000","last_unstale":"2024-09-16T11:15:21.483283+0000","last_undegraded":"2024-09-16T11:15:21.483283+0000","last_fullsized":"2024-09-16T11:15:21.483283+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:13.376287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674599999999998,"stat_sum":{"num_bytes":45944998,"num_objects":103,"num_object_clones":23,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":27,"num_read":18402,"num_read_kb":46297,"num_write":1139,"num_write_kb":20557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":9456611,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.542184+0000","last_change":"2024-09-16T11:13:52.391053+0000","last_active":"2024-09-16T11:15:20.542184+0000","last_peered":"2024-09-16T11:15:20.542184+0000","last_clean":"2024-09-16T11:15:20.542184+0000","last_became_active":"2024-09-16T11:13:52.390767+0000","last_became_peered":"2024-09-16T11:13:52.390767+0000","last_unstale":"2024-09-16T11:15:20.542184+0000","last_undegraded":"2024-09-16T11:15:20.542184+0000","last_fullsized":"2024-09-16T11:15:20.542184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:14.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"132'383","reported_seq":7601,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T11:15:17.513362+0000","last_change":"2024-09-16T11:15:17.512703+0000","last_active":"2024-09-16T11:15:17.513362+0000","last_peered":"2024-09-16T11:15:17.513362+0000","last_clean":"2024-09-16T11:15:17.513362+0000","last_became_active":"2024-09-16T11:15:17.508999+0000","last_became_peered":"2024-09-16T11:15:17.508999+0000","last_unstale":"2024-09-16T11:15:17.513362+0000","last_undegraded":"2024-09-16T11:15:17.513362+0000","last_fullsized":"2024-09-16T11:15:17.513362+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.390951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815220000000001,"stat_sum":{"num_bytes":25176905,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":21,"num_read":7319,"num_read_kb":28132,"num_write":488,"num_write_kb":10674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":27293022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.532462+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:15:20.532462+0000","last_peered":"2024-09-16T11:15:20.532462+0000","last_clean":"2024-09-16T11:15:20.532462+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:15:20.532462+0000","last_undegraded":"2024-09-16T11:15:20.532462+0000","last_fullsized":"2024-09-16T11:15:20.532462+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":54,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.546490+0000","last_change":"2024-09-16T11:14:55.442877+0000","last_active":"2024-09-16T11:15:20.546490+0000","last_peered":"2024-09-16T11:15:20.546490+0000","last_clean":"2024-09-16T11:15:20.546490+0000","last_became_active":"2024-09-16T11:14:55.438669+0000","last_became_peered":"2024-09-16T11:14:55.438669+0000","last_unstale":"2024-09-16T11:15:20.546490+0000","last_undegraded":"2024-09-16T11:15:20.546490+0000","last_fullsized":"2024-09-16T11:15:20.546490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:40.469472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"134'527","reported_seq":6137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.195538+0000","last_change":"2024-09-16T11:14:23.105967+0000","last_active":"2024-09-16T11:15:21.195538+0000","last_peered":"2024-09-16T11:15:21.195538+0000","last_clean":"2024-09-16T11:15:21.195538+0000","last_became_active":"2024-09-16T11:14:23.092519+0000","last_became_peered":"2024-09-16T11:14:23.092519+0000","last_unstale":"2024-09-16T11:15:21.195538+0000","last_undegraded":"2024-09-16T11:15:21.195538+0000","last_fullsized":"2024-09-16T11:15:21.195538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:58.815065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001093002,"stat_sum":{"num_bytes":65429730,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":14,"num_read":7273,"num_read_kb":59428,"num_write":897,"num_write_kb":29236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":14007837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570177+0000","last_change":"2024-09-16T11:14:20.060782+0000","last_active":"2024-09-16T11:15:20.570177+0000","last_peered":"2024-09-16T11:15:20.570177+0000","last_clean":"2024-09-16T11:15:20.570177+0000","last_became_active":"2024-09-16T11:14:20.060295+0000","last_became_peered":"2024-09-16T11:14:20.060295+0000","last_unstale":"2024-09-16T11:15:20.570177+0000","last_undegraded":"2024-09-16T11:15:20.570177+0000","last_fullsized":"2024-09-16T11:15:20.570177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:30.845736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570210+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:15:20.570210+0000","last_peered":"2024-09-16T11:15:20.570210+0000","last_clean":"2024-09-16T11:15:20.570210+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:15:20.570210+0000","last_undegraded":"2024-09-16T11:15:20.570210+0000","last_fullsized":"2024-09-16T11:15:20.570210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"127'515","reported_seq":3013,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T11:15:16.502804+0000","last_change":"2024-09-16T11:15:12.448451+0000","last_active":"2024-09-16T11:15:16.502804+0000","last_peered":"2024-09-16T11:15:16.502804+0000","last_clean":"2024-09-16T11:15:16.502804+0000","last_became_active":"2024-09-16T11:15:12.445855+0000","last_became_peered":"2024-09-16T11:15:12.445855+0000","last_unstale":"2024-09-16T11:15:16.502804+0000","last_undegraded":"2024-09-16T11:15:16.502804+0000","last_fullsized":"2024-09-16T11:15:16.502804+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:43:33.514680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014276930000000001,"stat_sum":{"num_bytes":18587238,"num_objects":85,"num_object_clones":22,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":23,"num_read":3445,"num_read_kb":9238,"num_write":643,"num_write_kb":16491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":4791707,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"51'1","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.542140+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:15:20.542140+0000","last_peered":"2024-09-16T11:15:20.542140+0000","last_clean":"2024-09-16T11:15:20.542140+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:15:20.542140+0000","last_undegraded":"2024-09-16T11:15:20.542140+0000","last_fullsized":"2024-09-16T11:15:20.542140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571020+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:15:20.571020+0000","last_peered":"2024-09-16T11:15:20.571020+0000","last_clean":"2024-09-16T11:15:20.571020+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:15:20.571020+0000","last_undegraded":"2024-09-16T11:15:20.571020+0000","last_fullsized":"2024-09-16T11:15:20.571020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"135'390","reported_seq":3168,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.539456+0000","last_change":"2024-09-16T11:13:15.331158+0000","last_active":"2024-09-16T11:15:21.539456+0000","last_peered":"2024-09-16T11:15:21.539456+0000","last_clean":"2024-09-16T11:15:21.539456+0000","last_became_active":"2024-09-16T11:13:13.110085+0000","last_became_peered":"2024-09-16T11:13:13.110085+0000","last_unstale":"2024-09-16T11:15:21.539456+0000","last_undegraded":"2024-09-16T11:15:21.539456+0000","last_fullsized":"2024-09-16T11:15:21.539456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:43.994646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220874,"stat_sum":{"num_bytes":57428616,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":19,"num_read":3271,"num_read_kb":98548,"num_write":431,"num_write_kb":53163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"96'2","reported_seq":135,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.529736+0000","last_change":"2024-09-16T11:14:55.446597+0000","last_active":"2024-09-16T11:15:20.529736+0000","last_peered":"2024-09-16T11:15:20.529736+0000","last_clean":"2024-09-16T11:15:20.529736+0000","last_became_active":"2024-09-16T11:14:55.438591+0000","last_became_peered":"2024-09-16T11:14:55.438591+0000","last_unstale":"2024-09-16T11:15:20.529736+0000","last_undegraded":"2024-09-16T11:15:20.529736+0000","last_fullsized":"2024-09-16T11:15:20.529736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:11:47.479160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":127,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570406+0000","last_change":"2024-09-16T11:14:55.442911+0000","last_active":"2024-09-16T11:15:20.570406+0000","last_peered":"2024-09-16T11:15:20.570406+0000","last_clean":"2024-09-16T11:15:20.570406+0000","last_became_active":"2024-09-16T11:14:55.440414+0000","last_became_peered":"2024-09-16T11:14:55.440414+0000","last_unstale":"2024-09-16T11:15:20.570406+0000","last_undegraded":"2024-09-16T11:15:20.570406+0000","last_fullsized":"2024-09-16T11:15:20.570406+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:20.765225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571084+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:15:20.571084+0000","last_peered":"2024-09-16T11:15:20.571084+0000","last_clean":"2024-09-16T11:15:20.571084+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:15:20.571084+0000","last_undegraded":"2024-09-16T11:15:20.571084+0000","last_fullsized":"2024-09-16T11:15:20.571084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"134'427","reported_seq":5817,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.428960+0000","last_change":"2024-09-16T11:15:06.451311+0000","last_active":"2024-09-16T11:15:21.428960+0000","last_peered":"2024-09-16T11:15:21.428960+0000","last_clean":"2024-09-16T11:15:21.428960+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:15:21.428960+0000","last_undegraded":"2024-09-16T11:15:21.428960+0000","last_fullsized":"2024-09-16T11:15:21.428960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059007199999999997,"stat_sum":{"num_bytes":37106195,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":14,"num_read":5992,"num_read_kb":32223,"num_write":467,"num_write_kb":15227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"134'754","reported_seq":2397,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.484097+0000","last_change":"2024-09-16T11:14:58.670074+0000","last_active":"2024-09-16T11:15:21.484097+0000","last_peered":"2024-09-16T11:15:21.484097+0000","last_clean":"2024-09-16T11:15:21.484097+0000","last_became_active":"2024-09-16T11:14:58.666316+0000","last_became_peered":"2024-09-16T11:14:58.666316+0000","last_unstale":"2024-09-16T11:15:21.484097+0000","last_undegraded":"2024-09-16T11:15:21.484097+0000","last_fullsized":"2024-09-16T11:15:21.484097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081336500000000003,"stat_sum":{"num_bytes":32998042,"num_objects":99,"num_object_clones":21,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":25,"num_read":3666,"num_read_kb":23300,"num_write":1072,"num_write_kb":11983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":8381723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"134'561","reported_seq":9417,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.485274+0000","last_change":"2024-09-16T11:15:06.450843+0000","last_active":"2024-09-16T11:15:21.485274+0000","last_peered":"2024-09-16T11:15:21.485274+0000","last_clean":"2024-09-16T11:15:21.485274+0000","last_became_active":"2024-09-16T11:15:06.443397+0000","last_became_peered":"2024-09-16T11:15:06.443397+0000","last_unstale":"2024-09-16T11:15:21.485274+0000","last_undegraded":"2024-09-16T11:15:21.485274+0000","last_fullsized":"2024-09-16T11:15:21.485274+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:45.235919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066151299999999997,"stat_sum":{"num_bytes":137920488,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":19,"num_read":10331,"num_read_kb":86013,"num_write":680,"num_write_kb":27385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":13213368,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"135'442","reported_seq":3903,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.549528+0000","last_change":"2024-09-16T11:13:15.134244+0000","last_active":"2024-09-16T11:15:21.549528+0000","last_peered":"2024-09-16T11:15:21.549528+0000","last_clean":"2024-09-16T11:15:21.549528+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:21.549528+0000","last_undegraded":"2024-09-16T11:15:21.549528+0000","last_fullsized":"2024-09-16T11:15:21.549528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:23.499921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054346600000000004,"stat_sum":{"num_bytes":49907739,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":20,"num_read":3727,"num_read_kb":45513,"num_write":504,"num_write_kb":19017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"135'499","reported_seq":6536,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.596579+0000","last_change":"2024-09-16T11:13:15.329809+0000","last_active":"2024-09-16T11:15:21.596579+0000","last_peered":"2024-09-16T11:15:21.596579+0000","last_clean":"2024-09-16T11:15:21.596579+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:21.596579+0000","last_undegraded":"2024-09-16T11:15:21.596579+0000","last_fullsized":"2024-09-16T11:15:21.596579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:36.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047034000000000003,"stat_sum":{"num_bytes":55331993,"num_objects":84,"num_object_clones":17,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":26,"num_read":7437,"num_read_kb":26259,"num_write":528,"num_write_kb":21818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"135'885","reported_seq":9735,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.580075+0000","last_change":"2024-09-16T11:15:04.705143+0000","last_active":"2024-09-16T11:15:21.580075+0000","last_peered":"2024-09-16T11:15:21.580075+0000","last_clean":"2024-09-16T11:15:21.580075+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:15:21.580075+0000","last_undegraded":"2024-09-16T11:15:21.580075+0000","last_fullsized":"2024-09-16T11:15:21.580075+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023754140000000002,"stat_sum":{"num_bytes":33271221,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":11677,"num_read_kb":40115,"num_write":1403,"num_write_kb":19323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"134'895","reported_seq":16891,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.481303+0000","last_change":"2024-09-16T11:15:09.477208+0000","last_active":"2024-09-16T11:15:21.481303+0000","last_peered":"2024-09-16T11:15:21.481303+0000","last_clean":"2024-09-16T11:15:21.481303+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:15:21.481303+0000","last_undegraded":"2024-09-16T11:15:21.481303+0000","last_fullsized":"2024-09-16T11:15:21.481303+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025819319999999999,"stat_sum":{"num_bytes":66109012,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":17009,"num_read_kb":66145,"num_write":1470,"num_write_kb":25023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"135'607","reported_seq":8220,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541540+0000","last_change":"2024-09-16T11:15:07.455688+0000","last_active":"2024-09-16T11:15:21.541540+0000","last_peered":"2024-09-16T11:15:21.541540+0000","last_clean":"2024-09-16T11:15:21.541540+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:15:21.541540+0000","last_undegraded":"2024-09-16T11:15:21.541540+0000","last_fullsized":"2024-09-16T11:15:21.541540+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00256719,"stat_sum":{"num_bytes":71642793,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":19,"num_read":9484,"num_read_kb":96898,"num_write":720,"num_write_kb":65893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"134'419","reported_seq":4430,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.305394+0000","last_change":"2024-09-16T11:14:44.551857+0000","last_active":"2024-09-16T11:15:21.305394+0000","last_peered":"2024-09-16T11:15:21.305394+0000","last_clean":"2024-09-16T11:15:21.305394+0000","last_became_active":"2024-09-16T11:14:44.547942+0000","last_became_peered":"2024-09-16T11:14:44.547942+0000","last_unstale":"2024-09-16T11:15:21.305394+0000","last_undegraded":"2024-09-16T11:15:21.305394+0000","last_fullsized":"2024-09-16T11:15:21.305394+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:05.099400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217035,"stat_sum":{"num_bytes":62751574,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":5023,"num_read_kb":23805,"num_write":467,"num_write_kb":27329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":14969107,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1735002244,"num_objects":2737,"num_object_clones":554,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":710,"num_read":239299,"num_read_kb":1408768,"num_write":24524,"num_write_kb":747004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4151,"num_bytes_recovered":483874655,"num_keys_recovered":573,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068433408,"data_stored":1158376645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18400,"ondisk_log_size":18400,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430996,"kb_used_data":129472,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93940844,"statfs":{"total":96636764160,"available":96195424256,"internally_reserved":0,"allocated":132579328,"data_stored":136507206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":1.026},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.77200000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1970000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.877,"15min":0.90100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.0669999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2110000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.995,"15min":1.006},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.83999999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.371,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.502,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.024,"5min":1.0629999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.873}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.125,"15min":1.131},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1240000000000001,"15min":1.1339999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.71099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449224,"kb_used_data":147708,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93922616,"statfs":{"total":96636764160,"available":96176758784,"internally_reserved":0,"allocated":151252992,"data_stored":164246993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2430000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.296,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.089,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.872,"15min":0.874},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.997,"15min":1.0149999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.198,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1539999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91200000000000003,"15min":0.90800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.353,"5min":1.466,"15min":1.466},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.014},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.371,"5min":1.417,"15min":1.417},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.22,"15min":1.1100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":15.597,"5min":15.597,"15min":15.597},"last":0.89500000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.298},{"interface":"front","average":{"1min":0.999,"5min":1.077,"15min":1.0900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5069999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.5,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.196},{"interface":"front","average":{"1min":1.069,"5min":1.077,"15min":1.079},"min":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.96299999999999997}]}]},{"osd":5,"up_from":0,"seq":55834574883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492336,"kb_used_data":182500,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93879504,"statfs":{"total":96636764160,"available":96132612096,"internally_reserved":0,"allocated":186880000,"data_stored":200750685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0409999999999999,"5min":1.329,"15min":1.329},"last":0.496},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.0049999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.88500000000000001,"15min":0.90700000000000003},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.149,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0129999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.93000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.034,"15min":1.0580000000000001},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1890000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.88900000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.879,"5min":1.0269999999999999,"15min":1.0509999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2250000000000001,"5min":2.077,"15min":2.077},"last":1.03}]},{"osd":5,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.085},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1200000000000001,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.093},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.073}]},{"osd":6,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.115,"15min":1.145},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3939999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.143,"15min":1.1499999999999999},"min":{"1min":0.54900000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.151}]},{"osd":7,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.0980000000000001,"15min":1.1359999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.173,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.242,"15min":1.282},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.496,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.1240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478536,"kb_used_data":181180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93893304,"statfs":{"total":96636764160,"available":96146743296,"internally_reserved":0,"allocated":185528320,"data_stored":199204863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1200000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.374}]},{"osd":2,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98799999999999999,"15min":0.997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.371,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0369999999999999,"15min":1.03},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.081,"15min":1.089},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0069999999999999,"15min":0.997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.623},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428324,"kb_used_data":118480,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93943516,"statfs":{"total":96636764160,"available":96198160384,"internally_reserved":0,"allocated":121323520,"data_stored":133807722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0980000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.28,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.496}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1240000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.64400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.496,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.133},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.238,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.36499999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0660000000000001},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.1659999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.075},"min":{"1min":0.65200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.369,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.744,"15min":1.744},"last":0.58499999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.119,"15min":1.127},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.589,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.373},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.304,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439736,"kb_used_data":129896,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93932104,"statfs":{"total":96636764160,"available":96186474496,"internally_reserved":0,"allocated":133013504,"data_stored":148762133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0249999999999999,"5min":1.877,"15min":1.877},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.82399999999999995,"15min":0.85199999999999998},"min":{"1min":0.40300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89600000000000002,"5min":1.98,"15min":1.98},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.71099999999999997,"15min":0.72799999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.96899999999999997,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.278,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78300000000000003,"15min":0.79700000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77000000000000002,"15min":0.78300000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.109,"5min":1.784,"15min":1.784},"last":0.64500000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94299999999999995,"15min":0.96199999999999997},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.155,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.044},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.306,"5min":2.597,"15min":2.597},"last":1.004}]},{"osd":5,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.091,"15min":1.1140000000000001},"min":{"1min":0.56699999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.4359999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0109999999999999,"15min":1.032},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.26,"5min":2.266,"15min":2.266},"last":1.0569999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.117,"15min":1.141},"min":{"1min":0.70499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.3799999999999999,"5min":2.552,"15min":2.552},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0960000000000001,"15min":1.1220000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.337,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.284}]},{"osd":7,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.103,"15min":1.111},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.4710000000000001,"5min":2.512,"15min":2.512},"last":1.246},{"interface":"front","average":{"1min":1.02,"5min":1.1519999999999999,"15min":1.1739999999999999},"min":{"1min":0.64300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.4790000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412732,"kb_used_data":115368,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93959108,"statfs":{"total":96636764160,"available":96214126592,"internally_reserved":0,"allocated":118136832,"data_stored":125730858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.034,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.82799999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.628},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4019999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.196},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":2.129,"15min":2.129},"last":1.022}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0980000000000001,"15min":1.1220000000000001},"min":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3420000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.05}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.026},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.397,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4670000000000001,"5min":1.841,"15min":1.841},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.034,"15min":1.042},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3839999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.248},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3600000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115924992,"data_stored":124337422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131391488,"data_stored":147959046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119705600,"data_stored":133004654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":183906304,"data_stored":198401712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":185262080,"data_stored":199947617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":52248576,"data_stored":56168601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":149626880,"data_stored":163443823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130367488,"data_stored":135113770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:15:26.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:15:26.845 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:26.842+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:26.865 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.865 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:26.933 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:26.926+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:26.975 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:26.976 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:26.984 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:27.033 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:27.030+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:27.083 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:27.084 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:27.090 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:27.134 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:27.135 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:27.130+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:27.139 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:15:27.149 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-16T11:15:26.434858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1735592714,"num_objects":2744,"num_object_clones":554,"num_object_copies":5488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2741,"num_whiteouts":712,"num_read":239365,"num_read_kb":1408823,"num_write":24628,"num_write_kb":748635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4158,"num_bytes_recovered":483874757,"num_keys_recovered":573,"num_objects_omap":44,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18478,"ondisk_log_size":18478,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":3131884,"kb_used_data":1004604,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":657470996,"statfs":{"total":676457349120,"available":673250299904,"internally_reserved":0,"allocated":1028714496,"data_stored":1109010460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.889394"},"pg_stats":[{"pgid":"4.1f","version":"134'354","reported_seq":5881,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479509+0000","last_change":"2024-09-16T11:14:58.670830+0000","last_active":"2024-09-16T11:15:21.479509+0000","last_peered":"2024-09-16T11:15:21.479509+0000","last_clean":"2024-09-16T11:15:21.479509+0000","last_became_active":"2024-09-16T11:14:58.669893+0000","last_became_peered":"2024-09-16T11:14:58.669893+0000","last_unstale":"2024-09-16T11:15:21.479509+0000","last_undegraded":"2024-09-16T11:15:21.479509+0000","last_fullsized":"2024-09-16T11:15:21.479509+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:03.555919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002295123,"stat_sum":{"num_bytes":32245286,"num_objects":93,"num_object_clones":23,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":26,"num_read":6299,"num_read_kb":31192,"num_write":332,"num_write_kb":11309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8822592,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"135'669","reported_seq":10535,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.539202+0000","last_change":"2024-09-16T11:13:15.329907+0000","last_active":"2024-09-16T11:15:21.539202+0000","last_peered":"2024-09-16T11:15:21.539202+0000","last_clean":"2024-09-16T11:15:21.539202+0000","last_became_active":"2024-09-16T11:13:13.129077+0000","last_became_peered":"2024-09-16T11:13:13.129077+0000","last_unstale":"2024-09-16T11:15:21.539202+0000","last_undegraded":"2024-09-16T11:15:21.539202+0000","last_fullsized":"2024-09-16T11:15:21.539202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:56.981990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050881799999999997,"stat_sum":{"num_bytes":67527044,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":21,"num_read":12400,"num_read_kb":91410,"num_write":861,"num_write_kb":42836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"134'408","reported_seq":4438,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.427703+0000","last_change":"2024-09-16T11:13:15.119814+0000","last_active":"2024-09-16T11:15:21.427703+0000","last_peered":"2024-09-16T11:15:21.427703+0000","last_clean":"2024-09-16T11:15:21.427703+0000","last_became_active":"2024-09-16T11:13:13.110835+0000","last_became_peered":"2024-09-16T11:13:13.110835+0000","last_unstale":"2024-09-16T11:15:21.427703+0000","last_undegraded":"2024-09-16T11:15:21.427703+0000","last_fullsized":"2024-09-16T11:15:21.427703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:25.687407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063997100000000003,"stat_sum":{"num_bytes":46346359,"num_objects":83,"num_object_clones":18,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":19,"num_read":4677,"num_read_kb":26057,"num_write":540,"num_write_kb":14465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"134'884","reported_seq":8133,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.305659+0000","last_change":"2024-09-16T11:14:24.176887+0000","last_active":"2024-09-16T11:15:21.305659+0000","last_peered":"2024-09-16T11:15:21.305659+0000","last_clean":"2024-09-16T11:15:21.305659+0000","last_became_active":"2024-09-16T11:14:24.105273+0000","last_became_peered":"2024-09-16T11:14:24.105273+0000","last_unstale":"2024-09-16T11:15:21.305659+0000","last_undegraded":"2024-09-16T11:15:21.305659+0000","last_fullsized":"2024-09-16T11:15:21.305659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:10.103253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038046360000000001,"stat_sum":{"num_bytes":40917279,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":26,"num_read":10975,"num_read_kb":37510,"num_write":1275,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2431957,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"134'420","reported_seq":7136,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.486025+0000","last_change":"2024-09-16T11:15:07.454596+0000","last_active":"2024-09-16T11:15:21.486025+0000","last_peered":"2024-09-16T11:15:21.486025+0000","last_clean":"2024-09-16T11:15:21.486025+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:15:21.486025+0000","last_undegraded":"2024-09-16T11:15:21.486025+0000","last_fullsized":"2024-09-16T11:15:21.486025+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029419009999999998,"stat_sum":{"num_bytes":20659487,"num_objects":76,"num_object_clones":12,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":18,"num_read":7224,"num_read_kb":24218,"num_write":558,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"134'556","reported_seq":7250,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.560894+0000","last_change":"2024-09-16T11:15:05.439505+0000","last_active":"2024-09-16T11:15:21.560894+0000","last_peered":"2024-09-16T11:15:21.560894+0000","last_clean":"2024-09-16T11:15:21.560894+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:15:21.560894+0000","last_undegraded":"2024-09-16T11:15:21.560894+0000","last_fullsized":"2024-09-16T11:15:21.560894+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024433810000000001,"stat_sum":{"num_bytes":45458930,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":18,"num_read":7891,"num_read_kb":40418,"num_write":734,"num_write_kb":21939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"134'663","reported_seq":2719,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.196724+0000","last_change":"2024-09-16T11:14:23.090655+0000","last_active":"2024-09-16T11:15:21.196724+0000","last_peered":"2024-09-16T11:15:21.196724+0000","last_clean":"2024-09-16T11:15:21.196724+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:15:21.196724+0000","last_undegraded":"2024-09-16T11:15:21.196724+0000","last_fullsized":"2024-09-16T11:15:21.196724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072097299999999997,"stat_sum":{"num_bytes":54271658,"num_objects":99,"num_object_clones":23,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":29,"num_read":4451,"num_read_kb":20695,"num_write":1023,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"134'609","reported_seq":4830,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541174+0000","last_change":"2024-09-16T11:15:08.474104+0000","last_active":"2024-09-16T11:15:21.541174+0000","last_peered":"2024-09-16T11:15:21.541174+0000","last_clean":"2024-09-16T11:15:21.541174+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:15:21.541174+0000","last_undegraded":"2024-09-16T11:15:21.541174+0000","last_fullsized":"2024-09-16T11:15:21.541174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055416799999999996,"stat_sum":{"num_bytes":84203863,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":17,"num_read":6285,"num_read_kb":35627,"num_write":701,"num_write_kb":29109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"134'1467","reported_seq":6095,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.526860+0000","last_change":"2024-09-16T11:13:15.120697+0000","last_active":"2024-09-16T11:15:21.526860+0000","last_peered":"2024-09-16T11:15:21.526860+0000","last_clean":"2024-09-16T11:15:21.526860+0000","last_became_active":"2024-09-16T11:13:13.131852+0000","last_became_peered":"2024-09-16T11:13:13.131852+0000","last_unstale":"2024-09-16T11:15:21.526860+0000","last_undegraded":"2024-09-16T11:15:21.526860+0000","last_fullsized":"2024-09-16T11:15:21.526860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1467,"log_dups_size":0,"ondisk_log_size":1467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:23.813033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041000900000000002,"stat_sum":{"num_bytes":26038056,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":13,"num_read":11635,"num_read_kb":16377,"num_write":2567,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"134'372","reported_seq":7041,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.192242+0000","last_change":"2024-09-16T11:13:15.330097+0000","last_active":"2024-09-16T11:15:21.192242+0000","last_peered":"2024-09-16T11:15:21.192242+0000","last_clean":"2024-09-16T11:15:21.192242+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:15:21.192242+0000","last_undegraded":"2024-09-16T11:15:21.192242+0000","last_fullsized":"2024-09-16T11:15:21.192242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061357700000000002,"stat_sum":{"num_bytes":59976669,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":22,"num_read":7444,"num_read_kb":49605,"num_write":444,"num_write_kb":16154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"134'368","reported_seq":3465,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479999+0000","last_change":"2024-09-16T11:14:58.670333+0000","last_active":"2024-09-16T11:15:21.479999+0000","last_peered":"2024-09-16T11:15:21.479999+0000","last_clean":"2024-09-16T11:15:21.479999+0000","last_became_active":"2024-09-16T11:14:58.668312+0000","last_became_peered":"2024-09-16T11:14:58.668312+0000","last_unstale":"2024-09-16T11:15:21.479999+0000","last_undegraded":"2024-09-16T11:15:21.479999+0000","last_fullsized":"2024-09-16T11:15:21.479999+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:09:03.441602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065955699999999996,"stat_sum":{"num_bytes":47454015,"num_objects":86,"num_object_clones":18,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":30,"num_read":3239,"num_read_kb":16381,"num_write":343,"num_write_kb":26293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":9395442,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"134'533","reported_seq":5932,"reported_epoch":135,"state":"peering","last_fresh":"2024-09-16T11:15:21.547318+0000","last_change":"2024-09-16T11:15:21.547592+0000","last_active":"2024-09-16T11:15:21.547297+0000","last_peered":"2024-09-16T11:15:21.356909+0000","last_clean":"2024-09-16T11:15:21.356909+0000","last_became_active":"2024-09-16T11:15:17.513082+0000","last_became_peered":"2024-09-16T11:15:17.513082+0000","last_unstale":"2024-09-16T11:15:21.547318+0000","last_undegraded":"2024-09-16T11:15:21.547318+0000","last_fullsized":"2024-09-16T11:15:21.547318+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:43:51.413401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066315600000000003,"stat_sum":{"num_bytes":82278726,"num_objects":87,"num_object_clones":22,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":26,"num_read":6953,"num_read_kb":59576,"num_write":511,"num_write_kb":27647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":8536088,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"135'440","reported_seq":3953,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541635+0000","last_change":"2024-09-16T11:15:05.443267+0000","last_active":"2024-09-16T11:15:21.541635+0000","last_peered":"2024-09-16T11:15:21.541635+0000","last_clean":"2024-09-16T11:15:21.541635+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:15:21.541635+0000","last_undegraded":"2024-09-16T11:15:21.541635+0000","last_fullsized":"2024-09-16T11:15:21.541635+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028235669999999999,"stat_sum":{"num_bytes":46436955,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":16,"num_read":4462,"num_read_kb":38147,"num_write":479,"num_write_kb":23350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"127'357","reported_seq":4747,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T11:15:16.506804+0000","last_change":"2024-09-16T11:15:11.447651+0000","last_active":"2024-09-16T11:15:16.506804+0000","last_peered":"2024-09-16T11:15:16.506804+0000","last_clean":"2024-09-16T11:15:16.506804+0000","last_became_active":"2024-09-16T11:15:11.445111+0000","last_became_peered":"2024-09-16T11:15:11.445111+0000","last_unstale":"2024-09-16T11:15:16.506804+0000","last_undegraded":"2024-09-16T11:15:16.506804+0000","last_fullsized":"2024-09-16T11:15:16.506804+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:42.682703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145994,"stat_sum":{"num_bytes":84902425,"num_objects":85,"num_object_clones":18,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":19,"num_read":4502,"num_read_kb":52887,"num_write":603,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":15771908,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"134'550","reported_seq":6717,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479323+0000","last_change":"2024-09-16T11:13:15.119821+0000","last_active":"2024-09-16T11:15:21.479323+0000","last_peered":"2024-09-16T11:15:21.479323+0000","last_clean":"2024-09-16T11:15:21.479323+0000","last_became_active":"2024-09-16T11:13:13.131936+0000","last_became_peered":"2024-09-16T11:13:13.131936+0000","last_unstale":"2024-09-16T11:15:21.479323+0000","last_undegraded":"2024-09-16T11:15:21.479323+0000","last_fullsized":"2024-09-16T11:15:21.479323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:24.053000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059135899999999996,"stat_sum":{"num_bytes":101583801,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":7906,"num_read_kb":96144,"num_write":620,"num_write_kb":44786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"135'498","reported_seq":6630,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.546518+0000","last_change":"2024-09-16T11:14:45.570341+0000","last_active":"2024-09-16T11:15:21.546518+0000","last_peered":"2024-09-16T11:15:21.546518+0000","last_clean":"2024-09-16T11:15:21.546518+0000","last_became_active":"2024-09-16T11:14:45.569559+0000","last_became_peered":"2024-09-16T11:14:45.569559+0000","last_unstale":"2024-09-16T11:15:21.546518+0000","last_undegraded":"2024-09-16T11:15:21.546518+0000","last_fullsized":"2024-09-16T11:15:21.546518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:51.205615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050522900000000005,"stat_sum":{"num_bytes":50584171,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":24,"num_read":7334,"num_read_kb":37445,"num_write":682,"num_write_kb":22939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":18357825,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570472+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:15:20.570472+0000","last_peered":"2024-09-16T11:15:20.570472+0000","last_clean":"2024-09-16T11:15:20.570472+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:15:20.570472+0000","last_undegraded":"2024-09-16T11:15:20.570472+0000","last_fullsized":"2024-09-16T11:15:20.570472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"134'786","reported_seq":4933,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.344084+0000","last_change":"2024-09-16T11:15:08.474134+0000","last_active":"2024-09-16T11:15:21.344084+0000","last_peered":"2024-09-16T11:15:21.344084+0000","last_clean":"2024-09-16T11:15:21.344084+0000","last_became_active":"2024-09-16T11:15:08.472016+0000","last_became_peered":"2024-09-16T11:15:08.472016+0000","last_unstale":"2024-09-16T11:15:21.344084+0000","last_undegraded":"2024-09-16T11:15:21.344084+0000","last_fullsized":"2024-09-16T11:15:21.344084+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:25.422198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071029199999999995,"stat_sum":{"num_bytes":36064192,"num_objects":88,"num_object_clones":17,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":28,"num_read":7145,"num_read_kb":17284,"num_write":995,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":12894750,"num_keys_recovered":93,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":129,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.546596+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:15:20.546596+0000","last_peered":"2024-09-16T11:15:20.546596+0000","last_clean":"2024-09-16T11:15:20.546596+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:15:20.546596+0000","last_undegraded":"2024-09-16T11:15:20.546596+0000","last_fullsized":"2024-09-16T11:15:20.546596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"134'332","reported_seq":4449,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.567960+0000","last_change":"2024-09-16T11:13:15.119575+0000","last_active":"2024-09-16T11:15:20.567960+0000","last_peered":"2024-09-16T11:15:20.567960+0000","last_clean":"2024-09-16T11:15:20.567960+0000","last_became_active":"2024-09-16T11:13:13.131951+0000","last_became_peered":"2024-09-16T11:13:13.131951+0000","last_unstale":"2024-09-16T11:15:20.567960+0000","last_undegraded":"2024-09-16T11:15:20.567960+0000","last_fullsized":"2024-09-16T11:15:20.567960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:26.039221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051524000000000003,"stat_sum":{"num_bytes":48446784,"num_objects":91,"num_object_clones":19,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":28,"num_read":4421,"num_read_kb":35881,"num_write":347,"num_write_kb":11458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571366+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:15:20.571366+0000","last_peered":"2024-09-16T11:15:20.571366+0000","last_clean":"2024-09-16T11:15:20.571366+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:15:20.571366+0000","last_undegraded":"2024-09-16T11:15:20.571366+0000","last_fullsized":"2024-09-16T11:15:20.571366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"134'830","reported_seq":16427,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.483283+0000","last_change":"2024-09-16T11:14:58.670274+0000","last_active":"2024-09-16T11:15:21.483283+0000","last_peered":"2024-09-16T11:15:21.483283+0000","last_clean":"2024-09-16T11:15:21.483283+0000","last_became_active":"2024-09-16T11:14:58.669120+0000","last_became_peered":"2024-09-16T11:14:58.669120+0000","last_unstale":"2024-09-16T11:15:21.483283+0000","last_undegraded":"2024-09-16T11:15:21.483283+0000","last_fullsized":"2024-09-16T11:15:21.483283+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:13.376287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674599999999998,"stat_sum":{"num_bytes":45944998,"num_objects":103,"num_object_clones":23,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":27,"num_read":18402,"num_read_kb":46297,"num_write":1139,"num_write_kb":20557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":9456611,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.542184+0000","last_change":"2024-09-16T11:13:52.391053+0000","last_active":"2024-09-16T11:15:20.542184+0000","last_peered":"2024-09-16T11:15:20.542184+0000","last_clean":"2024-09-16T11:15:20.542184+0000","last_became_active":"2024-09-16T11:13:52.390767+0000","last_became_peered":"2024-09-16T11:13:52.390767+0000","last_unstale":"2024-09-16T11:15:20.542184+0000","last_undegraded":"2024-09-16T11:15:20.542184+0000","last_fullsized":"2024-09-16T11:15:20.542184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:14.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"132'383","reported_seq":7601,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T11:15:17.513362+0000","last_change":"2024-09-16T11:15:17.512703+0000","last_active":"2024-09-16T11:15:17.513362+0000","last_peered":"2024-09-16T11:15:17.513362+0000","last_clean":"2024-09-16T11:15:17.513362+0000","last_became_active":"2024-09-16T11:15:17.508999+0000","last_became_peered":"2024-09-16T11:15:17.508999+0000","last_unstale":"2024-09-16T11:15:17.513362+0000","last_undegraded":"2024-09-16T11:15:17.513362+0000","last_fullsized":"2024-09-16T11:15:17.513362+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.390951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815220000000001,"stat_sum":{"num_bytes":25176905,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":21,"num_read":7319,"num_read_kb":28132,"num_write":488,"num_write_kb":10674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":27293022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.532462+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:15:20.532462+0000","last_peered":"2024-09-16T11:15:20.532462+0000","last_clean":"2024-09-16T11:15:20.532462+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:15:20.532462+0000","last_undegraded":"2024-09-16T11:15:20.532462+0000","last_fullsized":"2024-09-16T11:15:20.532462+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":54,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.546490+0000","last_change":"2024-09-16T11:14:55.442877+0000","last_active":"2024-09-16T11:15:20.546490+0000","last_peered":"2024-09-16T11:15:20.546490+0000","last_clean":"2024-09-16T11:15:20.546490+0000","last_became_active":"2024-09-16T11:14:55.438669+0000","last_became_peered":"2024-09-16T11:14:55.438669+0000","last_unstale":"2024-09-16T11:15:20.546490+0000","last_undegraded":"2024-09-16T11:15:20.546490+0000","last_fullsized":"2024-09-16T11:15:20.546490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:40.469472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"134'527","reported_seq":6137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.195538+0000","last_change":"2024-09-16T11:14:23.105967+0000","last_active":"2024-09-16T11:15:21.195538+0000","last_peered":"2024-09-16T11:15:21.195538+0000","last_clean":"2024-09-16T11:15:21.195538+0000","last_became_active":"2024-09-16T11:14:23.092519+0000","last_became_peered":"2024-09-16T11:14:23.092519+0000","last_unstale":"2024-09-16T11:15:21.195538+0000","last_undegraded":"2024-09-16T11:15:21.195538+0000","last_fullsized":"2024-09-16T11:15:21.195538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:58.815065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001093002,"stat_sum":{"num_bytes":65429730,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":14,"num_read":7273,"num_read_kb":59428,"num_write":897,"num_write_kb":29236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":14007837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570177+0000","last_change":"2024-09-16T11:14:20.060782+0000","last_active":"2024-09-16T11:15:20.570177+0000","last_peered":"2024-09-16T11:15:20.570177+0000","last_clean":"2024-09-16T11:15:20.570177+0000","last_became_active":"2024-09-16T11:14:20.060295+0000","last_became_peered":"2024-09-16T11:14:20.060295+0000","last_unstale":"2024-09-16T11:15:20.570177+0000","last_undegraded":"2024-09-16T11:15:20.570177+0000","last_fullsized":"2024-09-16T11:15:20.570177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:30.845736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570210+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:15:20.570210+0000","last_peered":"2024-09-16T11:15:20.570210+0000","last_clean":"2024-09-16T11:15:20.570210+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:15:20.570210+0000","last_undegraded":"2024-09-16T11:15:20.570210+0000","last_fullsized":"2024-09-16T11:15:20.570210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"127'515","reported_seq":3013,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T11:15:16.502804+0000","last_change":"2024-09-16T11:15:12.448451+0000","last_active":"2024-09-16T11:15:16.502804+0000","last_peered":"2024-09-16T11:15:16.502804+0000","last_clean":"2024-09-16T11:15:16.502804+0000","last_became_active":"2024-09-16T11:15:12.445855+0000","last_became_peered":"2024-09-16T11:15:12.445855+0000","last_unstale":"2024-09-16T11:15:16.502804+0000","last_undegraded":"2024-09-16T11:15:16.502804+0000","last_fullsized":"2024-09-16T11:15:16.502804+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:43:33.514680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014276930000000001,"stat_sum":{"num_bytes":18587238,"num_objects":85,"num_object_clones":22,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":23,"num_read":3445,"num_read_kb":9238,"num_write":643,"num_write_kb":16491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":4791707,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"51'1","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.542140+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:15:20.542140+0000","last_peered":"2024-09-16T11:15:20.542140+0000","last_clean":"2024-09-16T11:15:20.542140+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:15:20.542140+0000","last_undegraded":"2024-09-16T11:15:20.542140+0000","last_fullsized":"2024-09-16T11:15:20.542140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571020+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:15:20.571020+0000","last_peered":"2024-09-16T11:15:20.571020+0000","last_clean":"2024-09-16T11:15:20.571020+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:15:20.571020+0000","last_undegraded":"2024-09-16T11:15:20.571020+0000","last_fullsized":"2024-09-16T11:15:20.571020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"135'390","reported_seq":3168,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.539456+0000","last_change":"2024-09-16T11:13:15.331158+0000","last_active":"2024-09-16T11:15:21.539456+0000","last_peered":"2024-09-16T11:15:21.539456+0000","last_clean":"2024-09-16T11:15:21.539456+0000","last_became_active":"2024-09-16T11:13:13.110085+0000","last_became_peered":"2024-09-16T11:13:13.110085+0000","last_unstale":"2024-09-16T11:15:21.539456+0000","last_undegraded":"2024-09-16T11:15:21.539456+0000","last_fullsized":"2024-09-16T11:15:21.539456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:43.994646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220874,"stat_sum":{"num_bytes":57428616,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":19,"num_read":3271,"num_read_kb":98548,"num_write":431,"num_write_kb":53163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"96'2","reported_seq":135,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.529736+0000","last_change":"2024-09-16T11:14:55.446597+0000","last_active":"2024-09-16T11:15:20.529736+0000","last_peered":"2024-09-16T11:15:20.529736+0000","last_clean":"2024-09-16T11:15:20.529736+0000","last_became_active":"2024-09-16T11:14:55.438591+0000","last_became_peered":"2024-09-16T11:14:55.438591+0000","last_unstale":"2024-09-16T11:15:20.529736+0000","last_undegraded":"2024-09-16T11:15:20.529736+0000","last_fullsized":"2024-09-16T11:15:20.529736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:11:47.479160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":127,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570406+0000","last_change":"2024-09-16T11:14:55.442911+0000","last_active":"2024-09-16T11:15:20.570406+0000","last_peered":"2024-09-16T11:15:20.570406+0000","last_clean":"2024-09-16T11:15:20.570406+0000","last_became_active":"2024-09-16T11:14:55.440414+0000","last_became_peered":"2024-09-16T11:14:55.440414+0000","last_unstale":"2024-09-16T11:15:20.570406+0000","last_undegraded":"2024-09-16T11:15:20.570406+0000","last_fullsized":"2024-09-16T11:15:20.570406+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:20.765225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571084+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:15:20.571084+0000","last_peered":"2024-09-16T11:15:20.571084+0000","last_clean":"2024-09-16T11:15:20.571084+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:15:20.571084+0000","last_undegraded":"2024-09-16T11:15:20.571084+0000","last_fullsized":"2024-09-16T11:15:20.571084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"134'427","reported_seq":5817,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.428960+0000","last_change":"2024-09-16T11:15:06.451311+0000","last_active":"2024-09-16T11:15:21.428960+0000","last_peered":"2024-09-16T11:15:21.428960+0000","last_clean":"2024-09-16T11:15:21.428960+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:15:21.428960+0000","last_undegraded":"2024-09-16T11:15:21.428960+0000","last_fullsized":"2024-09-16T11:15:21.428960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059007199999999997,"stat_sum":{"num_bytes":37106195,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":14,"num_read":5992,"num_read_kb":32223,"num_write":467,"num_write_kb":15227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"134'754","reported_seq":2397,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.484097+0000","last_change":"2024-09-16T11:14:58.670074+0000","last_active":"2024-09-16T11:15:21.484097+0000","last_peered":"2024-09-16T11:15:21.484097+0000","last_clean":"2024-09-16T11:15:21.484097+0000","last_became_active":"2024-09-16T11:14:58.666316+0000","last_became_peered":"2024-09-16T11:14:58.666316+0000","last_unstale":"2024-09-16T11:15:21.484097+0000","last_undegraded":"2024-09-16T11:15:21.484097+0000","last_fullsized":"2024-09-16T11:15:21.484097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081336500000000003,"stat_sum":{"num_bytes":32998042,"num_objects":99,"num_object_clones":21,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":25,"num_read":3666,"num_read_kb":23300,"num_write":1072,"num_write_kb":11983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":8381723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"134'561","reported_seq":9417,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.485274+0000","last_change":"2024-09-16T11:15:06.450843+0000","last_active":"2024-09-16T11:15:21.485274+0000","last_peered":"2024-09-16T11:15:21.485274+0000","last_clean":"2024-09-16T11:15:21.485274+0000","last_became_active":"2024-09-16T11:15:06.443397+0000","last_became_peered":"2024-09-16T11:15:06.443397+0000","last_unstale":"2024-09-16T11:15:21.485274+0000","last_undegraded":"2024-09-16T11:15:21.485274+0000","last_fullsized":"2024-09-16T11:15:21.485274+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:45.235919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066151299999999997,"stat_sum":{"num_bytes":137920488,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":19,"num_read":10331,"num_read_kb":86013,"num_write":680,"num_write_kb":27385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":13213368,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"135'442","reported_seq":3903,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.549528+0000","last_change":"2024-09-16T11:13:15.134244+0000","last_active":"2024-09-16T11:15:21.549528+0000","last_peered":"2024-09-16T11:15:21.549528+0000","last_clean":"2024-09-16T11:15:21.549528+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:21.549528+0000","last_undegraded":"2024-09-16T11:15:21.549528+0000","last_fullsized":"2024-09-16T11:15:21.549528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:23.499921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054346600000000004,"stat_sum":{"num_bytes":49907739,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":20,"num_read":3727,"num_read_kb":45513,"num_write":504,"num_write_kb":19017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"135'499","reported_seq":6536,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.596579+0000","last_change":"2024-09-16T11:13:15.329809+0000","last_active":"2024-09-16T11:15:21.596579+0000","last_peered":"2024-09-16T11:15:21.596579+0000","last_clean":"2024-09-16T11:15:21.596579+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:21.596579+0000","last_undegraded":"2024-09-16T11:15:21.596579+0000","last_fullsized":"2024-09-16T11:15:21.596579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:36.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047034000000000003,"stat_sum":{"num_bytes":55331993,"num_objects":84,"num_object_clones":17,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":26,"num_read":7437,"num_read_kb":26259,"num_write":528,"num_write_kb":21818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"135'885","reported_seq":9735,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.580075+0000","last_change":"2024-09-16T11:15:04.705143+0000","last_active":"2024-09-16T11:15:21.580075+0000","last_peered":"2024-09-16T11:15:21.580075+0000","last_clean":"2024-09-16T11:15:21.580075+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:15:21.580075+0000","last_undegraded":"2024-09-16T11:15:21.580075+0000","last_fullsized":"2024-09-16T11:15:21.580075+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023754140000000002,"stat_sum":{"num_bytes":33271221,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":11677,"num_read_kb":40115,"num_write":1403,"num_write_kb":19323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"134'895","reported_seq":16891,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.481303+0000","last_change":"2024-09-16T11:15:09.477208+0000","last_active":"2024-09-16T11:15:21.481303+0000","last_peered":"2024-09-16T11:15:21.481303+0000","last_clean":"2024-09-16T11:15:21.481303+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:15:21.481303+0000","last_undegraded":"2024-09-16T11:15:21.481303+0000","last_fullsized":"2024-09-16T11:15:21.481303+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025819319999999999,"stat_sum":{"num_bytes":66109012,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":17009,"num_read_kb":66145,"num_write":1470,"num_write_kb":25023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"135'607","reported_seq":8220,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541540+0000","last_change":"2024-09-16T11:15:07.455688+0000","last_active":"2024-09-16T11:15:21.541540+0000","last_peered":"2024-09-16T11:15:21.541540+0000","last_clean":"2024-09-16T11:15:21.541540+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:15:21.541540+0000","last_undegraded":"2024-09-16T11:15:21.541540+0000","last_fullsized":"2024-09-16T11:15:21.541540+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00256719,"stat_sum":{"num_bytes":71642793,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":19,"num_read":9484,"num_read_kb":96898,"num_write":720,"num_write_kb":65893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"134'419","reported_seq":4430,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.305394+0000","last_change":"2024-09-16T11:14:44.551857+0000","last_active":"2024-09-16T11:15:21.305394+0000","last_peered":"2024-09-16T11:15:21.305394+0000","last_clean":"2024-09-16T11:15:21.305394+0000","last_became_active":"2024-09-16T11:14:44.547942+0000","last_became_peered":"2024-09-16T11:14:44.547942+0000","last_unstale":"2024-09-16T11:15:21.305394+0000","last_undegraded":"2024-09-16T11:15:21.305394+0000","last_fullsized":"2024-09-16T11:15:21.305394+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:05.099400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217035,"stat_sum":{"num_bytes":62751574,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":5023,"num_read_kb":23805,"num_write":467,"num_write_kb":27329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":14969107,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1735002244,"num_objects":2737,"num_object_clones":554,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":710,"num_read":239299,"num_read_kb":1408768,"num_write":24524,"num_write_kb":747004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4151,"num_bytes_recovered":483874655,"num_keys_recovered":573,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068433408,"data_stored":1158376645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18400,"ondisk_log_size":18400,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430996,"kb_used_data":129472,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93940844,"statfs":{"total":96636764160,"available":96195424256,"internally_reserved":0,"allocated":132579328,"data_stored":136507206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":1.026},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.77200000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1970000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.877,"15min":0.90100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.0669999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2110000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.995,"15min":1.006},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.83999999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.371,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.502,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.024,"5min":1.0629999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.873}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.125,"15min":1.131},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1240000000000001,"15min":1.1339999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.71099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449224,"kb_used_data":147708,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93922616,"statfs":{"total":96636764160,"available":96176758784,"internally_reserved":0,"allocated":151252992,"data_stored":164246993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2430000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.296,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.089,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.872,"15min":0.874},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.997,"15min":1.0149999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.198,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1539999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91200000000000003,"15min":0.90800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.353,"5min":1.466,"15min":1.466},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.014},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.371,"5min":1.417,"15min":1.417},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.22,"15min":1.1100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":15.597,"5min":15.597,"15min":15.597},"last":0.89500000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.298},{"interface":"front","average":{"1min":0.999,"5min":1.077,"15min":1.0900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5069999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.5,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.196},{"interface":"front","average":{"1min":1.069,"5min":1.077,"15min":1.079},"min":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.96299999999999997}]}]},{"osd":5,"up_from":0,"seq":55834574883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492336,"kb_used_data":182500,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93879504,"statfs":{"total":96636764160,"available":96132612096,"internally_reserved":0,"allocated":186880000,"data_stored":200750685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0409999999999999,"5min":1.329,"15min":1.329},"last":0.496},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.0049999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.88500000000000001,"15min":0.90700000000000003},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.149,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0129999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.93000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.034,"15min":1.0580000000000001},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1890000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.88900000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.879,"5min":1.0269999999999999,"15min":1.0509999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2250000000000001,"5min":2.077,"15min":2.077},"last":1.03}]},{"osd":5,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.085},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1200000000000001,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.093},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.073}]},{"osd":6,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.115,"15min":1.145},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3939999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.143,"15min":1.1499999999999999},"min":{"1min":0.54900000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.151}]},{"osd":7,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.0980000000000001,"15min":1.1359999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.173,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.242,"15min":1.282},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.496,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.1240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478536,"kb_used_data":181180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93893304,"statfs":{"total":96636764160,"available":96146743296,"internally_reserved":0,"allocated":185528320,"data_stored":199204863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1200000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.374}]},{"osd":2,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98799999999999999,"15min":0.997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.371,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0369999999999999,"15min":1.03},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.081,"15min":1.089},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0069999999999999,"15min":0.997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.623},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428324,"kb_used_data":118480,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93943516,"statfs":{"total":96636764160,"available":96198160384,"internally_reserved":0,"allocated":121323520,"data_stored":133807722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0980000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.28,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.496}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1240000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.64400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.496,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.133},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.238,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.36499999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0660000000000001},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.1659999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.075},"min":{"1min":0.65200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.369,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.744,"15min":1.744},"last":0.58499999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.119,"15min":1.127},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.589,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.373},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.304,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439736,"kb_used_data":129896,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93932104,"statfs":{"total":96636764160,"available":96186474496,"internally_reserved":0,"allocated":133013504,"data_stored":148762133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0249999999999999,"5min":1.877,"15min":1.877},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.82399999999999995,"15min":0.85199999999999998},"min":{"1min":0.40300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89600000000000002,"5min":1.98,"15min":1.98},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.71099999999999997,"15min":0.72799999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.96899999999999997,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.278,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78300000000000003,"15min":0.79700000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77000000000000002,"15min":0.78300000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.109,"5min":1.784,"15min":1.784},"last":0.64500000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94299999999999995,"15min":0.96199999999999997},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.155,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.044},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.306,"5min":2.597,"15min":2.597},"last":1.004}]},{"osd":5,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.091,"15min":1.1140000000000001},"min":{"1min":0.56699999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.4359999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0109999999999999,"15min":1.032},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.26,"5min":2.266,"15min":2.266},"last":1.0569999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.117,"15min":1.141},"min":{"1min":0.70499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.3799999999999999,"5min":2.552,"15min":2.552},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0960000000000001,"15min":1.1220000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.337,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.284}]},{"osd":7,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.103,"15min":1.111},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.4710000000000001,"5min":2.512,"15min":2.512},"last":1.246},{"interface":"front","average":{"1min":1.02,"5min":1.1519999999999999,"15min":1.1739999999999999},"min":{"1min":0.64300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.4790000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412732,"kb_used_data":115368,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93959108,"statfs":{"total":96636764160,"available":96214126592,"internally_reserved":0,"allocated":118136832,"data_stored":125730858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.034,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.82799999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.628},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4019999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.196},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":2.129,"15min":2.129},"last":1.022}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0980000000000001,"15min":1.1220000000000001},"min":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3420000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.05}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.026},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.397,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4670000000000001,"5min":1.841,"15min":1.841},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.034,"15min":1.042},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3839999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.248},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3600000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115924992,"data_stored":124337422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131391488,"data_stored":147959046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119705600,"data_stored":133004654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":183906304,"data_stored":198401712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":185262080,"data_stored":199947617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":52248576,"data_stored":56168601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":149626880,"data_stored":163443823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130367488,"data_stored":135113770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:15:27.152 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T11:15:27.189 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:27.190 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:27.198 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:27.235 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:27.233+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:27.300 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:27.300 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:27.309 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:27.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:27.354 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:27.353+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:27.424 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:27.425 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:27.434 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:27.436 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:27.430+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:27.437 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:27.463 INFO:teuthology.orchestra.run.smithi162.stdout:5851 write 0x39c6494 thru 0x39d54af (0xf01c bytes) 2024-09-16T11:15:27.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:15:27.467 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:27.480 INFO:teuthology.orchestra.run.smithi162.stdout:5852 trunc from 0xe6b2526 to 0x9745fac 2024-09-16T11:15:27.498 INFO:teuthology.orchestra.run.smithi162.stdout:5853 punch from 0xa3b3d0 to 0xa49d85, (0xe9b5 bytes) 2024-09-16T11:15:27.498 INFO:teuthology.orchestra.run.smithi162.stdout:5854 write 0x9364178 thru 0x936e51f (0xa3a8 bytes) 2024-09-16T11:15:27.500 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:27.507 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:27.507 INFO:teuthology.orchestra.run.smithi162.stdout:5856 read 0xc0197f thru 0xc05b03 (0x4185 bytes) 2024-09-16T11:15:27.508 INFO:teuthology.orchestra.run.smithi162.stdout:5857 trunc from 0x9745fac to 0x8dfb3da 2024-09-16T11:15:27.533 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:27.537 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:27.530+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:27.542 INFO:teuthology.orchestra.run.smithi162.stdout:5859 write 0xbf14e13 thru 0xbf1ddaf (0x8f9d bytes) 2024-09-16T11:15:27.547 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:27.547 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:27.547 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:27.547 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:27.547 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:27.553 INFO:teuthology.orchestra.run.smithi162.stdout:5861 write 0x7c5bccd thru 0x7c5de8b (0x21bf bytes) 2024-09-16T11:15:27.557 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:27.564 INFO:teuthology.orchestra.run.smithi162.stdout:5862 write 0x3ae957a thru 0x3af0f50 (0x79d7 bytes) 2024-09-16T11:15:27.565 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:27.575 INFO:teuthology.orchestra.run.smithi162.stdout:5864 write 0x3d67167 thru 0x3d6c6cd (0x5567 bytes) 2024-09-16T11:15:27.588 INFO:teuthology.orchestra.run.smithi162.stdout:5865 write 0x4f9cfc thru 0x508f17 (0xf21c bytes) 2024-09-16T11:15:27.591 INFO:teuthology.orchestra.run.smithi162.stdout:5866 punch from 0xb673b37 to 0xb680f80, (0xd449 bytes) 2024-09-16T11:15:27.591 INFO:teuthology.orchestra.run.smithi162.stdout:5867 read 0x4053de6 thru 0x405ac50 (0x6e6b bytes) 2024-09-16T11:15:27.592 INFO:teuthology.orchestra.run.smithi162.stdout:5870 writesame 0x7b52535 thru 0x7b5e06e data_size 0x5(0xbb3a bytes) 2024-09-16T11:15:27.597 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:27.601 INFO:teuthology.orchestra.run.smithi162.stdout:5872 trunc from 0xbf1ddb0 to 0xe6fc851 2024-09-16T11:15:27.622 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:27.630 INFO:teuthology.orchestra.run.smithi162.stdout:5873 trunc from 0xe6fc851 to 0x3ba3ea2 2024-09-16T11:15:27.639 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:27.637+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:27.644 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:27.644 INFO:teuthology.orchestra.run.smithi162.stdout:{"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":172,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":141,"num_osds":8,"num_up_osds":8,"osd_up_since":1726485166,"num_in_osds":7,"osd_in_since":1726485300,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":44},{"state_name":"peering","count":1}],"num_pgs":45,"num_pools":4,"num_objects":2744,"data_bytes":1735592714,"bytes_used":3207049216,"bytes_avail":673250299904,"bytes_total":676457349120,"inactive_pgs_ratio":0.02222222276031971,"read_bytes_sec":4022296,"write_bytes_sec":2486166,"read_op_per_sec":2087,"write_op_per_sec":517},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-16T11:13:47.959553+0000","services":{}},"progress_events":{}} 2024-09-16T11:15:27.651 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:27.654 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:27.658 INFO:teuthology.orchestra.run.smithi162.stdout:5874 read 0x2307a70 thru 0x2315d3c (0xe2cd bytes) 2024-09-16T11:15:27.659 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T11:15:27.660 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:15:27.660 INFO:teuthology.orchestra.run.smithi162.stdout:5878 trunc from 0x3ba3ea2 to 0x354cc54 2024-09-16T11:15:27.677 INFO:teuthology.orchestra.run.smithi162.stdout:5879 read 0x10b9b57 thru 0x10bb653 (0x1afd bytes) 2024-09-16T11:15:27.678 INFO:teuthology.orchestra.run.smithi162.stdout:5880 trunc from 0x354cc54 to 0x23e6995 2024-09-16T11:15:27.692 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.514933687, 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42183 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 622592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:27.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.512470413, 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063470921, 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42183 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.176741+0000", 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177852+0000", 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0011116590000000001 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177858+0000", 2024-09-16T11:15:27.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3770000000000004e-06 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177867+0000", 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8019999999999998e-06 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177872+0000", 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6160000000000001e-06 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239699+0000", 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.061827103000000001 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.239802+0000", 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102718 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.240212+0000", 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000409646 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.701 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.511262178, 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.702 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:27.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.706 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.509196886, 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.50918303, 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:27.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.509169155, 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:27.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:27.719 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.509155388, 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.720 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:27.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:27.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.509107879, 2024-09-16T11:15:27.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:27.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.728 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.508822061, 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:27.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.733 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.41615824500001, 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:27.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 103.416009063, 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.740 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:27.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:27.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:27.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427607843000001, 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.746 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:27.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:27.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427560883000002, 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:27.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:27.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:27.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.754 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427542191000001, 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.755 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427525992, 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.759 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.763 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427510949999999, 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:27.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427495199999999, 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.768 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:27.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427479909999999, 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:27.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:27.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427464332, 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:27.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:27.781 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.427449353, 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:27.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:27.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:27.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:27.786 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:27.787 INFO:teuthology.orchestra.run.smithi162.stdout:5881 punch from 0x1b73812 to 0x1b73b18, (0x306 bytes) 2024-09-16T11:15:27.787 INFO:teuthology.orchestra.run.smithi162.stdout:5882 read 0x2034b9b thru 0x203d11b (0x8581 bytes) 2024-09-16T11:15:27.787 INFO:teuthology.orchestra.run.smithi162.stdout:5884 read 0xa279ae thru 0xa28824 (0xe77 bytes) 2024-09-16T11:15:27.788 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:27.789 INFO:teuthology.orchestra.run.smithi162.stdout:5885 write 0xb5d52ff thru 0xb5d8d55 (0x3a57 bytes) 2024-09-16T11:15:27.789 INFO:teuthology.orchestra.run.smithi162.stdout:5886 read 0x1c2f274 thru 0x1c3ba16 (0xc7a3 bytes) 2024-09-16T11:15:27.789 INFO:teuthology.orchestra.run.smithi162.stdout:5888 punch from 0x652e7ca to 0x652ef3a, (0x770 bytes) 2024-09-16T11:15:27.789 INFO:teuthology.orchestra.run.smithi162.stdout:5889 write 0x1228e08 thru 0x1232ac2 (0x9cbb bytes) 2024-09-16T11:15:27.790 INFO:teuthology.orchestra.run.smithi162.stdout:5890 punch from 0xf6f3ae to 0xf71dce, (0x2a20 bytes) 2024-09-16T11:15:27.790 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:27.790 INFO:teuthology.orchestra.run.smithi162.stdout:5893 read 0x4387fb6 thru 0x438c702 (0x474d bytes) 2024-09-16T11:15:27.790 INFO:teuthology.orchestra.run.smithi162.stdout:5894 clone 42 order 25 su 1048576 sc 9 2024-09-16T11:15:27.790 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:27.782+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:27.889 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:27.885+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:27.926 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:27.926 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:27.926 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:27.926 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:27.935 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:27.966 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:27.968 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:15:27.978 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-16T11:15:27.438861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1735592714,"num_objects":2744,"num_object_clones":554,"num_object_copies":5488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2741,"num_whiteouts":712,"num_read":239365,"num_read_kb":1408823,"num_write":24628,"num_write_kb":748635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4158,"num_bytes_recovered":483874757,"num_keys_recovered":573,"num_objects_omap":44,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18478,"ondisk_log_size":18478,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":3131884,"kb_used_data":1004604,"kb_used_omap":11,"kb_used_meta":2127156,"kb_avail":657470996,"statfs":{"total":676457349120,"available":673250299904,"internally_reserved":0,"allocated":1028714496,"data_stored":1109010460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2178208104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.888475"},"pg_stats":[{"pgid":"4.1f","version":"134'354","reported_seq":5881,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479509+0000","last_change":"2024-09-16T11:14:58.670830+0000","last_active":"2024-09-16T11:15:21.479509+0000","last_peered":"2024-09-16T11:15:21.479509+0000","last_clean":"2024-09-16T11:15:21.479509+0000","last_became_active":"2024-09-16T11:14:58.669893+0000","last_became_peered":"2024-09-16T11:14:58.669893+0000","last_unstale":"2024-09-16T11:15:21.479509+0000","last_undegraded":"2024-09-16T11:15:21.479509+0000","last_fullsized":"2024-09-16T11:15:21.479509+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:03.555919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002295123,"stat_sum":{"num_bytes":32245286,"num_objects":93,"num_object_clones":23,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":26,"num_read":6299,"num_read_kb":31192,"num_write":332,"num_write_kb":11309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8822592,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"135'669","reported_seq":10535,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.539202+0000","last_change":"2024-09-16T11:13:15.329907+0000","last_active":"2024-09-16T11:15:21.539202+0000","last_peered":"2024-09-16T11:15:21.539202+0000","last_clean":"2024-09-16T11:15:21.539202+0000","last_became_active":"2024-09-16T11:13:13.129077+0000","last_became_peered":"2024-09-16T11:13:13.129077+0000","last_unstale":"2024-09-16T11:15:21.539202+0000","last_undegraded":"2024-09-16T11:15:21.539202+0000","last_fullsized":"2024-09-16T11:15:21.539202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:56.981990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050881799999999997,"stat_sum":{"num_bytes":67527044,"num_objects":79,"num_object_clones":11,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":21,"num_read":12400,"num_read_kb":91410,"num_write":861,"num_write_kb":42836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"134'408","reported_seq":4438,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.427703+0000","last_change":"2024-09-16T11:13:15.119814+0000","last_active":"2024-09-16T11:15:21.427703+0000","last_peered":"2024-09-16T11:15:21.427703+0000","last_clean":"2024-09-16T11:15:21.427703+0000","last_became_active":"2024-09-16T11:13:13.110835+0000","last_became_peered":"2024-09-16T11:13:13.110835+0000","last_unstale":"2024-09-16T11:15:21.427703+0000","last_undegraded":"2024-09-16T11:15:21.427703+0000","last_fullsized":"2024-09-16T11:15:21.427703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:25.687407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063997100000000003,"stat_sum":{"num_bytes":46346359,"num_objects":83,"num_object_clones":18,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":19,"num_read":4677,"num_read_kb":26057,"num_write":540,"num_write_kb":14465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"134'884","reported_seq":8133,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.305659+0000","last_change":"2024-09-16T11:14:24.176887+0000","last_active":"2024-09-16T11:15:21.305659+0000","last_peered":"2024-09-16T11:15:21.305659+0000","last_clean":"2024-09-16T11:15:21.305659+0000","last_became_active":"2024-09-16T11:14:24.105273+0000","last_became_peered":"2024-09-16T11:14:24.105273+0000","last_unstale":"2024-09-16T11:15:21.305659+0000","last_undegraded":"2024-09-16T11:15:21.305659+0000","last_fullsized":"2024-09-16T11:15:21.305659+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:10.103253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038046360000000001,"stat_sum":{"num_bytes":40917279,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":26,"num_read":10975,"num_read_kb":37510,"num_write":1275,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2431957,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"134'420","reported_seq":7136,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.486025+0000","last_change":"2024-09-16T11:15:07.454596+0000","last_active":"2024-09-16T11:15:21.486025+0000","last_peered":"2024-09-16T11:15:21.486025+0000","last_clean":"2024-09-16T11:15:21.486025+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:15:21.486025+0000","last_undegraded":"2024-09-16T11:15:21.486025+0000","last_fullsized":"2024-09-16T11:15:21.486025+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029419009999999998,"stat_sum":{"num_bytes":20659487,"num_objects":76,"num_object_clones":12,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":18,"num_read":7224,"num_read_kb":24218,"num_write":558,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"134'556","reported_seq":7250,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.560894+0000","last_change":"2024-09-16T11:15:05.439505+0000","last_active":"2024-09-16T11:15:21.560894+0000","last_peered":"2024-09-16T11:15:21.560894+0000","last_clean":"2024-09-16T11:15:21.560894+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:15:21.560894+0000","last_undegraded":"2024-09-16T11:15:21.560894+0000","last_fullsized":"2024-09-16T11:15:21.560894+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024433810000000001,"stat_sum":{"num_bytes":45458930,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":18,"num_read":7891,"num_read_kb":40418,"num_write":734,"num_write_kb":21939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"134'663","reported_seq":2719,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.196724+0000","last_change":"2024-09-16T11:14:23.090655+0000","last_active":"2024-09-16T11:15:21.196724+0000","last_peered":"2024-09-16T11:15:21.196724+0000","last_clean":"2024-09-16T11:15:21.196724+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:15:21.196724+0000","last_undegraded":"2024-09-16T11:15:21.196724+0000","last_fullsized":"2024-09-16T11:15:21.196724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072097299999999997,"stat_sum":{"num_bytes":54271658,"num_objects":99,"num_object_clones":23,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":29,"num_read":4451,"num_read_kb":20695,"num_write":1023,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"134'609","reported_seq":4830,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541174+0000","last_change":"2024-09-16T11:15:08.474104+0000","last_active":"2024-09-16T11:15:21.541174+0000","last_peered":"2024-09-16T11:15:21.541174+0000","last_clean":"2024-09-16T11:15:21.541174+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:15:21.541174+0000","last_undegraded":"2024-09-16T11:15:21.541174+0000","last_fullsized":"2024-09-16T11:15:21.541174+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055416799999999996,"stat_sum":{"num_bytes":84203863,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":17,"num_read":6285,"num_read_kb":35627,"num_write":701,"num_write_kb":29109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"134'1467","reported_seq":6095,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.526860+0000","last_change":"2024-09-16T11:13:15.120697+0000","last_active":"2024-09-16T11:15:21.526860+0000","last_peered":"2024-09-16T11:15:21.526860+0000","last_clean":"2024-09-16T11:15:21.526860+0000","last_became_active":"2024-09-16T11:13:13.131852+0000","last_became_peered":"2024-09-16T11:13:13.131852+0000","last_unstale":"2024-09-16T11:15:21.526860+0000","last_undegraded":"2024-09-16T11:15:21.526860+0000","last_fullsized":"2024-09-16T11:15:21.526860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1467,"log_dups_size":0,"ondisk_log_size":1467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:23.813033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041000900000000002,"stat_sum":{"num_bytes":26038056,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":13,"num_read":11635,"num_read_kb":16377,"num_write":2567,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"134'372","reported_seq":7041,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.192242+0000","last_change":"2024-09-16T11:13:15.330097+0000","last_active":"2024-09-16T11:15:21.192242+0000","last_peered":"2024-09-16T11:15:21.192242+0000","last_clean":"2024-09-16T11:15:21.192242+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:15:21.192242+0000","last_undegraded":"2024-09-16T11:15:21.192242+0000","last_fullsized":"2024-09-16T11:15:21.192242+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061357700000000002,"stat_sum":{"num_bytes":59976669,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":22,"num_read":7444,"num_read_kb":49605,"num_write":444,"num_write_kb":16154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"134'368","reported_seq":3465,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479999+0000","last_change":"2024-09-16T11:14:58.670333+0000","last_active":"2024-09-16T11:15:21.479999+0000","last_peered":"2024-09-16T11:15:21.479999+0000","last_clean":"2024-09-16T11:15:21.479999+0000","last_became_active":"2024-09-16T11:14:58.668312+0000","last_became_peered":"2024-09-16T11:14:58.668312+0000","last_unstale":"2024-09-16T11:15:21.479999+0000","last_undegraded":"2024-09-16T11:15:21.479999+0000","last_fullsized":"2024-09-16T11:15:21.479999+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:09:03.441602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065955699999999996,"stat_sum":{"num_bytes":47454015,"num_objects":86,"num_object_clones":18,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":30,"num_read":3239,"num_read_kb":16381,"num_write":343,"num_write_kb":26293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":9395442,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"134'533","reported_seq":5932,"reported_epoch":135,"state":"peering","last_fresh":"2024-09-16T11:15:21.547318+0000","last_change":"2024-09-16T11:15:21.547592+0000","last_active":"2024-09-16T11:15:21.547297+0000","last_peered":"2024-09-16T11:15:21.356909+0000","last_clean":"2024-09-16T11:15:21.356909+0000","last_became_active":"2024-09-16T11:15:17.513082+0000","last_became_peered":"2024-09-16T11:15:17.513082+0000","last_unstale":"2024-09-16T11:15:21.547318+0000","last_undegraded":"2024-09-16T11:15:21.547318+0000","last_fullsized":"2024-09-16T11:15:21.547318+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:43:51.413401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066315600000000003,"stat_sum":{"num_bytes":82278726,"num_objects":87,"num_object_clones":22,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":26,"num_read":6953,"num_read_kb":59576,"num_write":511,"num_write_kb":27647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":8536088,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"135'440","reported_seq":3953,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541635+0000","last_change":"2024-09-16T11:15:05.443267+0000","last_active":"2024-09-16T11:15:21.541635+0000","last_peered":"2024-09-16T11:15:21.541635+0000","last_clean":"2024-09-16T11:15:21.541635+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:15:21.541635+0000","last_undegraded":"2024-09-16T11:15:21.541635+0000","last_fullsized":"2024-09-16T11:15:21.541635+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028235669999999999,"stat_sum":{"num_bytes":46436955,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":16,"num_read":4462,"num_read_kb":38147,"num_write":479,"num_write_kb":23350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"127'357","reported_seq":4747,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T11:15:16.506804+0000","last_change":"2024-09-16T11:15:11.447651+0000","last_active":"2024-09-16T11:15:16.506804+0000","last_peered":"2024-09-16T11:15:16.506804+0000","last_clean":"2024-09-16T11:15:16.506804+0000","last_became_active":"2024-09-16T11:15:11.445111+0000","last_became_peered":"2024-09-16T11:15:11.445111+0000","last_unstale":"2024-09-16T11:15:16.506804+0000","last_undegraded":"2024-09-16T11:15:16.506804+0000","last_fullsized":"2024-09-16T11:15:16.506804+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:42.682703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145994,"stat_sum":{"num_bytes":84902425,"num_objects":85,"num_object_clones":18,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":19,"num_read":4502,"num_read_kb":52887,"num_write":603,"num_write_kb":23202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":15771908,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"134'550","reported_seq":6717,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.479323+0000","last_change":"2024-09-16T11:13:15.119821+0000","last_active":"2024-09-16T11:15:21.479323+0000","last_peered":"2024-09-16T11:15:21.479323+0000","last_clean":"2024-09-16T11:15:21.479323+0000","last_became_active":"2024-09-16T11:13:13.131936+0000","last_became_peered":"2024-09-16T11:13:13.131936+0000","last_unstale":"2024-09-16T11:15:21.479323+0000","last_undegraded":"2024-09-16T11:15:21.479323+0000","last_fullsized":"2024-09-16T11:15:21.479323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:24.053000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059135899999999996,"stat_sum":{"num_bytes":101583801,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":30,"num_read":7906,"num_read_kb":96144,"num_write":620,"num_write_kb":44786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"135'498","reported_seq":6630,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.546518+0000","last_change":"2024-09-16T11:14:45.570341+0000","last_active":"2024-09-16T11:15:21.546518+0000","last_peered":"2024-09-16T11:15:21.546518+0000","last_clean":"2024-09-16T11:15:21.546518+0000","last_became_active":"2024-09-16T11:14:45.569559+0000","last_became_peered":"2024-09-16T11:14:45.569559+0000","last_unstale":"2024-09-16T11:15:21.546518+0000","last_undegraded":"2024-09-16T11:15:21.546518+0000","last_fullsized":"2024-09-16T11:15:21.546518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:51.205615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050522900000000005,"stat_sum":{"num_bytes":50584171,"num_objects":86,"num_object_clones":16,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":24,"num_read":7334,"num_read_kb":37445,"num_write":682,"num_write_kb":22939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":18357825,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570472+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:15:20.570472+0000","last_peered":"2024-09-16T11:15:20.570472+0000","last_clean":"2024-09-16T11:15:20.570472+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:15:20.570472+0000","last_undegraded":"2024-09-16T11:15:20.570472+0000","last_fullsized":"2024-09-16T11:15:20.570472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"134'786","reported_seq":4933,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.344084+0000","last_change":"2024-09-16T11:15:08.474134+0000","last_active":"2024-09-16T11:15:21.344084+0000","last_peered":"2024-09-16T11:15:21.344084+0000","last_clean":"2024-09-16T11:15:21.344084+0000","last_became_active":"2024-09-16T11:15:08.472016+0000","last_became_peered":"2024-09-16T11:15:08.472016+0000","last_unstale":"2024-09-16T11:15:21.344084+0000","last_undegraded":"2024-09-16T11:15:21.344084+0000","last_fullsized":"2024-09-16T11:15:21.344084+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:25.422198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071029199999999995,"stat_sum":{"num_bytes":36064192,"num_objects":88,"num_object_clones":17,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":28,"num_read":7145,"num_read_kb":17284,"num_write":995,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":12894750,"num_keys_recovered":93,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":129,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.546596+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:15:20.546596+0000","last_peered":"2024-09-16T11:15:20.546596+0000","last_clean":"2024-09-16T11:15:20.546596+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:15:20.546596+0000","last_undegraded":"2024-09-16T11:15:20.546596+0000","last_fullsized":"2024-09-16T11:15:20.546596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"134'332","reported_seq":4449,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.567960+0000","last_change":"2024-09-16T11:13:15.119575+0000","last_active":"2024-09-16T11:15:20.567960+0000","last_peered":"2024-09-16T11:15:20.567960+0000","last_clean":"2024-09-16T11:15:20.567960+0000","last_became_active":"2024-09-16T11:13:13.131951+0000","last_became_peered":"2024-09-16T11:13:13.131951+0000","last_unstale":"2024-09-16T11:15:20.567960+0000","last_undegraded":"2024-09-16T11:15:20.567960+0000","last_fullsized":"2024-09-16T11:15:20.567960+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:26.039221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051524000000000003,"stat_sum":{"num_bytes":48446784,"num_objects":91,"num_object_clones":19,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":28,"num_read":4421,"num_read_kb":35881,"num_write":347,"num_write_kb":11458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571366+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:15:20.571366+0000","last_peered":"2024-09-16T11:15:20.571366+0000","last_clean":"2024-09-16T11:15:20.571366+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:15:20.571366+0000","last_undegraded":"2024-09-16T11:15:20.571366+0000","last_fullsized":"2024-09-16T11:15:20.571366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"134'830","reported_seq":16427,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.483283+0000","last_change":"2024-09-16T11:14:58.670274+0000","last_active":"2024-09-16T11:15:21.483283+0000","last_peered":"2024-09-16T11:15:21.483283+0000","last_clean":"2024-09-16T11:15:21.483283+0000","last_became_active":"2024-09-16T11:14:58.669120+0000","last_became_peered":"2024-09-16T11:14:58.669120+0000","last_unstale":"2024-09-16T11:15:21.483283+0000","last_undegraded":"2024-09-16T11:15:21.483283+0000","last_fullsized":"2024-09-16T11:15:21.483283+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:13.376287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674599999999998,"stat_sum":{"num_bytes":45944998,"num_objects":103,"num_object_clones":23,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":27,"num_read":18402,"num_read_kb":46297,"num_write":1139,"num_write_kb":20557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":9456611,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.542184+0000","last_change":"2024-09-16T11:13:52.391053+0000","last_active":"2024-09-16T11:15:20.542184+0000","last_peered":"2024-09-16T11:15:20.542184+0000","last_clean":"2024-09-16T11:15:20.542184+0000","last_became_active":"2024-09-16T11:13:52.390767+0000","last_became_peered":"2024-09-16T11:13:52.390767+0000","last_unstale":"2024-09-16T11:15:20.542184+0000","last_undegraded":"2024-09-16T11:15:20.542184+0000","last_fullsized":"2024-09-16T11:15:20.542184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:14.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"132'383","reported_seq":7601,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T11:15:17.513362+0000","last_change":"2024-09-16T11:15:17.512703+0000","last_active":"2024-09-16T11:15:17.513362+0000","last_peered":"2024-09-16T11:15:17.513362+0000","last_clean":"2024-09-16T11:15:17.513362+0000","last_became_active":"2024-09-16T11:15:17.508999+0000","last_became_peered":"2024-09-16T11:15:17.508999+0000","last_unstale":"2024-09-16T11:15:17.513362+0000","last_undegraded":"2024-09-16T11:15:17.513362+0000","last_fullsized":"2024-09-16T11:15:17.513362+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.390951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815220000000001,"stat_sum":{"num_bytes":25176905,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":21,"num_read":7319,"num_read_kb":28132,"num_write":488,"num_write_kb":10674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":27293022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.532462+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:15:20.532462+0000","last_peered":"2024-09-16T11:15:20.532462+0000","last_clean":"2024-09-16T11:15:20.532462+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:15:20.532462+0000","last_undegraded":"2024-09-16T11:15:20.532462+0000","last_fullsized":"2024-09-16T11:15:20.532462+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":54,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.546490+0000","last_change":"2024-09-16T11:14:55.442877+0000","last_active":"2024-09-16T11:15:20.546490+0000","last_peered":"2024-09-16T11:15:20.546490+0000","last_clean":"2024-09-16T11:15:20.546490+0000","last_became_active":"2024-09-16T11:14:55.438669+0000","last_became_peered":"2024-09-16T11:14:55.438669+0000","last_unstale":"2024-09-16T11:15:20.546490+0000","last_undegraded":"2024-09-16T11:15:20.546490+0000","last_fullsized":"2024-09-16T11:15:20.546490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:40.469472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"134'527","reported_seq":6137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.195538+0000","last_change":"2024-09-16T11:14:23.105967+0000","last_active":"2024-09-16T11:15:21.195538+0000","last_peered":"2024-09-16T11:15:21.195538+0000","last_clean":"2024-09-16T11:15:21.195538+0000","last_became_active":"2024-09-16T11:14:23.092519+0000","last_became_peered":"2024-09-16T11:14:23.092519+0000","last_unstale":"2024-09-16T11:15:21.195538+0000","last_undegraded":"2024-09-16T11:15:21.195538+0000","last_fullsized":"2024-09-16T11:15:21.195538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:58.815065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001093002,"stat_sum":{"num_bytes":65429730,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":14,"num_read":7273,"num_read_kb":59428,"num_write":897,"num_write_kb":29236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":14007837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570177+0000","last_change":"2024-09-16T11:14:20.060782+0000","last_active":"2024-09-16T11:15:20.570177+0000","last_peered":"2024-09-16T11:15:20.570177+0000","last_clean":"2024-09-16T11:15:20.570177+0000","last_became_active":"2024-09-16T11:14:20.060295+0000","last_became_peered":"2024-09-16T11:14:20.060295+0000","last_unstale":"2024-09-16T11:15:20.570177+0000","last_undegraded":"2024-09-16T11:15:20.570177+0000","last_fullsized":"2024-09-16T11:15:20.570177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:30.845736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570210+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:15:20.570210+0000","last_peered":"2024-09-16T11:15:20.570210+0000","last_clean":"2024-09-16T11:15:20.570210+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:15:20.570210+0000","last_undegraded":"2024-09-16T11:15:20.570210+0000","last_fullsized":"2024-09-16T11:15:20.570210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"127'515","reported_seq":3013,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T11:15:16.502804+0000","last_change":"2024-09-16T11:15:12.448451+0000","last_active":"2024-09-16T11:15:16.502804+0000","last_peered":"2024-09-16T11:15:16.502804+0000","last_clean":"2024-09-16T11:15:16.502804+0000","last_became_active":"2024-09-16T11:15:12.445855+0000","last_became_peered":"2024-09-16T11:15:12.445855+0000","last_unstale":"2024-09-16T11:15:16.502804+0000","last_undegraded":"2024-09-16T11:15:16.502804+0000","last_fullsized":"2024-09-16T11:15:16.502804+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:43:33.514680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014276930000000001,"stat_sum":{"num_bytes":18587238,"num_objects":85,"num_object_clones":22,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":23,"num_read":3445,"num_read_kb":9238,"num_write":643,"num_write_kb":16491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":4791707,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"51'1","reported_seq":145,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.542140+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:15:20.542140+0000","last_peered":"2024-09-16T11:15:20.542140+0000","last_clean":"2024-09-16T11:15:20.542140+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:15:20.542140+0000","last_undegraded":"2024-09-16T11:15:20.542140+0000","last_fullsized":"2024-09-16T11:15:20.542140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571020+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:15:20.571020+0000","last_peered":"2024-09-16T11:15:20.571020+0000","last_clean":"2024-09-16T11:15:20.571020+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:15:20.571020+0000","last_undegraded":"2024-09-16T11:15:20.571020+0000","last_fullsized":"2024-09-16T11:15:20.571020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"135'390","reported_seq":3168,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.539456+0000","last_change":"2024-09-16T11:13:15.331158+0000","last_active":"2024-09-16T11:15:21.539456+0000","last_peered":"2024-09-16T11:15:21.539456+0000","last_clean":"2024-09-16T11:15:21.539456+0000","last_became_active":"2024-09-16T11:13:13.110085+0000","last_became_peered":"2024-09-16T11:13:13.110085+0000","last_unstale":"2024-09-16T11:15:21.539456+0000","last_undegraded":"2024-09-16T11:15:21.539456+0000","last_fullsized":"2024-09-16T11:15:21.539456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:43.994646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220874,"stat_sum":{"num_bytes":57428616,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":19,"num_read":3271,"num_read_kb":98548,"num_write":431,"num_write_kb":53163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"96'2","reported_seq":135,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.529736+0000","last_change":"2024-09-16T11:14:55.446597+0000","last_active":"2024-09-16T11:15:20.529736+0000","last_peered":"2024-09-16T11:15:20.529736+0000","last_clean":"2024-09-16T11:15:20.529736+0000","last_became_active":"2024-09-16T11:14:55.438591+0000","last_became_peered":"2024-09-16T11:14:55.438591+0000","last_unstale":"2024-09-16T11:15:20.529736+0000","last_undegraded":"2024-09-16T11:15:20.529736+0000","last_fullsized":"2024-09-16T11:15:20.529736+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:11:47.479160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":127,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.570406+0000","last_change":"2024-09-16T11:14:55.442911+0000","last_active":"2024-09-16T11:15:20.570406+0000","last_peered":"2024-09-16T11:15:20.570406+0000","last_clean":"2024-09-16T11:15:20.570406+0000","last_became_active":"2024-09-16T11:14:55.440414+0000","last_became_peered":"2024-09-16T11:14:55.440414+0000","last_unstale":"2024-09-16T11:15:20.570406+0000","last_undegraded":"2024-09-16T11:15:20.570406+0000","last_fullsized":"2024-09-16T11:15:20.570406+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:20.765225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":216,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:20.571084+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:15:20.571084+0000","last_peered":"2024-09-16T11:15:20.571084+0000","last_clean":"2024-09-16T11:15:20.571084+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:15:20.571084+0000","last_undegraded":"2024-09-16T11:15:20.571084+0000","last_fullsized":"2024-09-16T11:15:20.571084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"134'427","reported_seq":5817,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.428960+0000","last_change":"2024-09-16T11:15:06.451311+0000","last_active":"2024-09-16T11:15:21.428960+0000","last_peered":"2024-09-16T11:15:21.428960+0000","last_clean":"2024-09-16T11:15:21.428960+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:15:21.428960+0000","last_undegraded":"2024-09-16T11:15:21.428960+0000","last_fullsized":"2024-09-16T11:15:21.428960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059007199999999997,"stat_sum":{"num_bytes":37106195,"num_objects":93,"num_object_clones":20,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":14,"num_read":5992,"num_read_kb":32223,"num_write":467,"num_write_kb":15227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"134'754","reported_seq":2397,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.484097+0000","last_change":"2024-09-16T11:14:58.670074+0000","last_active":"2024-09-16T11:15:21.484097+0000","last_peered":"2024-09-16T11:15:21.484097+0000","last_clean":"2024-09-16T11:15:21.484097+0000","last_became_active":"2024-09-16T11:14:58.666316+0000","last_became_peered":"2024-09-16T11:14:58.666316+0000","last_unstale":"2024-09-16T11:15:21.484097+0000","last_undegraded":"2024-09-16T11:15:21.484097+0000","last_fullsized":"2024-09-16T11:15:21.484097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081336500000000003,"stat_sum":{"num_bytes":32998042,"num_objects":99,"num_object_clones":21,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":25,"num_read":3666,"num_read_kb":23300,"num_write":1072,"num_write_kb":11983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":8381723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"134'561","reported_seq":9417,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.485274+0000","last_change":"2024-09-16T11:15:06.450843+0000","last_active":"2024-09-16T11:15:21.485274+0000","last_peered":"2024-09-16T11:15:21.485274+0000","last_clean":"2024-09-16T11:15:21.485274+0000","last_became_active":"2024-09-16T11:15:06.443397+0000","last_became_peered":"2024-09-16T11:15:06.443397+0000","last_unstale":"2024-09-16T11:15:21.485274+0000","last_undegraded":"2024-09-16T11:15:21.485274+0000","last_fullsized":"2024-09-16T11:15:21.485274+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:45.235919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066151299999999997,"stat_sum":{"num_bytes":137920488,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":19,"num_read":10331,"num_read_kb":86013,"num_write":680,"num_write_kb":27385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":13213368,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"135'442","reported_seq":3903,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.549528+0000","last_change":"2024-09-16T11:13:15.134244+0000","last_active":"2024-09-16T11:15:21.549528+0000","last_peered":"2024-09-16T11:15:21.549528+0000","last_clean":"2024-09-16T11:15:21.549528+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:21.549528+0000","last_undegraded":"2024-09-16T11:15:21.549528+0000","last_fullsized":"2024-09-16T11:15:21.549528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:23.499921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054346600000000004,"stat_sum":{"num_bytes":49907739,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":20,"num_read":3727,"num_read_kb":45513,"num_write":504,"num_write_kb":19017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"135'499","reported_seq":6536,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.596579+0000","last_change":"2024-09-16T11:13:15.329809+0000","last_active":"2024-09-16T11:15:21.596579+0000","last_peered":"2024-09-16T11:15:21.596579+0000","last_clean":"2024-09-16T11:15:21.596579+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:21.596579+0000","last_undegraded":"2024-09-16T11:15:21.596579+0000","last_fullsized":"2024-09-16T11:15:21.596579+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:36.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047034000000000003,"stat_sum":{"num_bytes":55331993,"num_objects":84,"num_object_clones":17,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":26,"num_read":7437,"num_read_kb":26259,"num_write":528,"num_write_kb":21818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"135'885","reported_seq":9735,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.580075+0000","last_change":"2024-09-16T11:15:04.705143+0000","last_active":"2024-09-16T11:15:21.580075+0000","last_peered":"2024-09-16T11:15:21.580075+0000","last_clean":"2024-09-16T11:15:21.580075+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:15:21.580075+0000","last_undegraded":"2024-09-16T11:15:21.580075+0000","last_fullsized":"2024-09-16T11:15:21.580075+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023754140000000002,"stat_sum":{"num_bytes":33271221,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":11677,"num_read_kb":40115,"num_write":1403,"num_write_kb":19323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"134'895","reported_seq":16891,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.481303+0000","last_change":"2024-09-16T11:15:09.477208+0000","last_active":"2024-09-16T11:15:21.481303+0000","last_peered":"2024-09-16T11:15:21.481303+0000","last_clean":"2024-09-16T11:15:21.481303+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:15:21.481303+0000","last_undegraded":"2024-09-16T11:15:21.481303+0000","last_fullsized":"2024-09-16T11:15:21.481303+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025819319999999999,"stat_sum":{"num_bytes":66109012,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":17009,"num_read_kb":66145,"num_write":1470,"num_write_kb":25023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"135'607","reported_seq":8220,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.541540+0000","last_change":"2024-09-16T11:15:07.455688+0000","last_active":"2024-09-16T11:15:21.541540+0000","last_peered":"2024-09-16T11:15:21.541540+0000","last_clean":"2024-09-16T11:15:21.541540+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:15:21.541540+0000","last_undegraded":"2024-09-16T11:15:21.541540+0000","last_fullsized":"2024-09-16T11:15:21.541540+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00256719,"stat_sum":{"num_bytes":71642793,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":19,"num_read":9484,"num_read_kb":96898,"num_write":720,"num_write_kb":65893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"134'419","reported_seq":4430,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T11:15:21.305394+0000","last_change":"2024-09-16T11:14:44.551857+0000","last_active":"2024-09-16T11:15:21.305394+0000","last_peered":"2024-09-16T11:15:21.305394+0000","last_clean":"2024-09-16T11:15:21.305394+0000","last_became_active":"2024-09-16T11:14:44.547942+0000","last_became_peered":"2024-09-16T11:14:44.547942+0000","last_unstale":"2024-09-16T11:15:21.305394+0000","last_undegraded":"2024-09-16T11:15:21.305394+0000","last_fullsized":"2024-09-16T11:15:21.305394+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:05.099400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217035,"stat_sum":{"num_bytes":62751574,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":23,"num_read":5023,"num_read_kb":23805,"num_write":467,"num_write_kb":27329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":14969107,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1735002244,"num_objects":2737,"num_object_clones":554,"num_object_copies":5474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2737,"num_whiteouts":710,"num_read":239299,"num_read_kb":1408768,"num_write":24524,"num_write_kb":747004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4151,"num_bytes_recovered":483874655,"num_keys_recovered":573,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068433408,"data_stored":1158376645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18400,"ondisk_log_size":18400,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430996,"kb_used_data":129472,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93940844,"statfs":{"total":96636764160,"available":96195424256,"internally_reserved":0,"allocated":132579328,"data_stored":136507206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":1.026},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.77200000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1970000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.877,"15min":0.90100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.0669999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2110000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.995,"15min":1.006},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.83999999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.371,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.502,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.024,"5min":1.0629999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":0.873}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.125,"15min":1.131},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1240000000000001,"15min":1.1339999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.71099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449224,"kb_used_data":147708,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93922616,"statfs":{"total":96636764160,"available":96176758784,"internally_reserved":0,"allocated":151252992,"data_stored":164246993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2430000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.296,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.089,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.872,"15min":0.874},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.997,"15min":1.0149999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.198,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1539999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91200000000000003,"15min":0.90800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.353,"5min":1.466,"15min":1.466},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.014},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.371,"5min":1.417,"15min":1.417},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.22,"15min":1.1100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":15.597,"5min":15.597,"15min":15.597},"last":0.89500000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.298},{"interface":"front","average":{"1min":0.999,"5min":1.077,"15min":1.0900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5069999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.5,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.196},{"interface":"front","average":{"1min":1.069,"5min":1.077,"15min":1.079},"min":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.96299999999999997}]}]},{"osd":5,"up_from":0,"seq":55834574883,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492336,"kb_used_data":182500,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93879504,"statfs":{"total":96636764160,"available":96132612096,"internally_reserved":0,"allocated":186880000,"data_stored":200750685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0409999999999999,"5min":1.329,"15min":1.329},"last":0.496},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.0049999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.88500000000000001,"15min":0.90700000000000003},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.149,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0129999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.93000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.034,"15min":1.0580000000000001},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1890000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.88900000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.879,"5min":1.0269999999999999,"15min":1.0509999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2250000000000001,"5min":2.077,"15min":2.077},"last":1.03}]},{"osd":5,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.085},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1200000000000001,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.093},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.073}]},{"osd":6,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.115,"15min":1.145},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3939999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.143,"15min":1.1499999999999999},"min":{"1min":0.54900000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.151}]},{"osd":7,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.0980000000000001,"15min":1.1359999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.173,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.242,"15min":1.282},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.496,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.1240000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478536,"kb_used_data":181180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93893304,"statfs":{"total":96636764160,"available":96146743296,"internally_reserved":0,"allocated":185528320,"data_stored":199204863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1200000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.374}]},{"osd":2,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98799999999999999,"15min":0.997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.371,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0369999999999999,"15min":1.03},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.081,"15min":1.089},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0069999999999999,"15min":0.997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.623},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428324,"kb_used_data":118480,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93943516,"statfs":{"total":96636764160,"available":96198160384,"internally_reserved":0,"allocated":121323520,"data_stored":133807722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0980000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.28,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.496}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1240000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.51400000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.64400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.496,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.133},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.238,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.36499999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0660000000000001},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":1.1659999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.075},"min":{"1min":0.65200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.369,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.744,"15min":1.744},"last":0.58499999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.119,"15min":1.127},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.589,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.373},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.304,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439736,"kb_used_data":129896,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93932104,"statfs":{"total":96636764160,"available":96186474496,"internally_reserved":0,"allocated":133013504,"data_stored":148762133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0249999999999999,"5min":1.877,"15min":1.877},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.82399999999999995,"15min":0.85199999999999998},"min":{"1min":0.40300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89600000000000002,"5min":1.98,"15min":1.98},"last":0.46700000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.71099999999999997,"15min":0.72799999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.96899999999999997,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.278,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78300000000000003,"15min":0.79700000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77000000000000002,"15min":0.78300000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.109,"5min":1.784,"15min":1.784},"last":0.64500000000000002}]},{"osd":4,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94299999999999995,"15min":0.96199999999999997},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.155,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.044},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.306,"5min":2.597,"15min":2.597},"last":1.004}]},{"osd":5,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.091,"15min":1.1140000000000001},"min":{"1min":0.56699999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.4359999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0109999999999999,"15min":1.032},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.26,"5min":2.266,"15min":2.266},"last":1.0569999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.117,"15min":1.141},"min":{"1min":0.70499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.3799999999999999,"5min":2.552,"15min":2.552},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0960000000000001,"15min":1.1220000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.337,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.284}]},{"osd":7,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.103,"15min":1.111},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.4710000000000001,"5min":2.512,"15min":2.512},"last":1.246},{"interface":"front","average":{"1min":1.02,"5min":1.1519999999999999,"15min":1.1739999999999999},"min":{"1min":0.64300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.4790000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412732,"kb_used_data":115368,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93959108,"statfs":{"total":96636764160,"available":96214126592,"internally_reserved":0,"allocated":118136832,"data_stored":125730858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.034,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.82799999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.628},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4019999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.1659999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.196},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":2.129,"15min":2.129},"last":1.022}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0980000000000001,"15min":1.1220000000000001},"min":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3420000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.05}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.026},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.397,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4670000000000001,"5min":1.841,"15min":1.841},"last":1.1240000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.034,"15min":1.042},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3839999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.248},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3600000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.0980000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115924992,"data_stored":124337422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131391488,"data_stored":147959046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119705600,"data_stored":133004654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":183906304,"data_stored":198401712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":185262080,"data_stored":199947617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":52248576,"data_stored":56168601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":149626880,"data_stored":163443823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":130367488,"data_stored":135113770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:15:27.990 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:27.989+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:28.034 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.034 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:28.035 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:28.035 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:28.035 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:28.043 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:28.091 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:28.086+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:28.146 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.146 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:28.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:28.146 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:28.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.146 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.648727500000007, 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:28.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:28.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:28.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.395314701999993, 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:28.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.395045386000007, 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:28.158 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:28.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:28.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.394646875999996, 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:28.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:28.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.394358088000004, 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:28.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.392658671999996, 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:28.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.392580355999996, 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:28.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.392546616999994, 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.392505872000001, 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:28.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:28.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.389305424, 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.191 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:28.195 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.389287366999994, 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:28.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.389261378, 2024-09-16T11:15:28.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:28.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:28.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.388562946999997, 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.205 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:28.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.388550387999999, 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.209 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:28.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.478874634999997, 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:28.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:28.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.796449129999999, 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:28.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:28.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:28.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.793584237000001, 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:28.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:28.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:28.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.784538320999999, 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.234 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:28.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:28.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.727122816000001, 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:28.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:28.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:28.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.680863650999999, 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:28.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.254 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:28.255 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:28.256 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:28.253+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:28.356 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:28.350+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:28.404 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:28.404 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:28.404 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:28.413 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:28.457 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:28.453+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:28.512 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.512 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:28.512 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:28.512 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:28.513 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:28.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:28.557 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:28.554+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:28.604 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:28.631 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550550321000003, 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:28.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:28.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:28.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550535862999993, 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.637 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:28.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:28.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550518663000005, 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.642 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:28.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:28.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:28.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.648 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550505142999995, 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:28.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:28.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550496601000006, 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:28.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.659 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550486457000005, 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:28.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:28.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.55047295, 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:28.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550459357999998, 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.670 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:28.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:28.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550435075999999, 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:28.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:28.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550400914999997, 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:28.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:28.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550367518000002, 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:28.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:28.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550344873, 2024-09-16T11:15:28.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:28.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:28.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550329297999994, 2024-09-16T11:15:28.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:28.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.700 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.550312263999999, 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.701 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:28.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:28.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.546679299999994, 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.706 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:28.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.546644958000002, 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:28.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.546629011999997, 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:28.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.546613401000002, 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.721 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:28.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.546598313000004, 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.726 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:28.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:28.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 66.546583510999994, 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:28.731 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:28.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:28.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:28.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:28.736 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:28.738 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:28.738 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:28.734+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:28.839 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:28.837+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:28.861 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-16T11:15:28.883 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.883 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:28.883 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:28.883 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:28.894 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:28.940 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:28.934+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:28.996 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:28.996 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:28.996 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:28.996 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:28.997 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:29.006 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:29.041 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:29.037+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.107 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.969235404, 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:29.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.969217016, 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:29.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.968916259, 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:29.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.120 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.96888655399999, 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:29.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:29.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.125 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.968873156, 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:29.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:29.129 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.968859352, 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.130 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.132 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.133 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:29.134 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:29.135 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.96848435299999, 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.136 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:29.137 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.138 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:29.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.954940322, 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.954918748, 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.954902249, 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.149 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:29.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.95488617300001, 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.95450259899999, 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.942420422, 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.942412221, 2024-09-16T11:15:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.94240419899999, 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:29.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.258925320000003, 2024-09-16T11:15:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.025506969000006, 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 57.639113596000001, 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 55.687508948000001, 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.551712016000003, 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:29.205 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:29.207 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:29.207 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:29.202+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:29.275 INFO:teuthology.orchestra.run.smithi162.stdout:5895 writesame 0xb75276a thru 0xb754f1f data_size 0x22(0x27b6 bytes) 2024-09-16T11:15:29.285 INFO:teuthology.orchestra.run.smithi162.stdout:5897 read 0x757746b thru 0x75775a1 (0x137 bytes) 2024-09-16T11:15:29.288 INFO:teuthology.orchestra.run.smithi162.stdout:5898 trunc from 0xb754f20 to 0xa3d1edf 2024-09-16T11:15:29.307 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:29.302+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:29.325 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:29.325 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:29.325 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:29.325 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:29.335 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:29.338 INFO:teuthology.orchestra.run.smithi162.stdout:5899 read 0x991117 thru 0x9a0d01 (0xfbeb bytes) 2024-09-16T11:15:29.353 INFO:teuthology.orchestra.run.smithi162.stdout:5901 write 0xb5bb5f7 thru 0xb5c4a0b (0x9415 bytes) 2024-09-16T11:15:29.357 INFO:teuthology.orchestra.run.smithi162.stdout:5902 writesame 0xa25474e thru 0xa261586 data_size 0x1f(0xce39 bytes) 2024-09-16T11:15:29.382 INFO:teuthology.orchestra.run.smithi162.stdout:5903 writesame 0xe5417f6 thru 0xe54dde1 data_size 0x4(0xc5ec bytes) 2024-09-16T11:15:29.385 INFO:teuthology.orchestra.run.smithi162.stdout:5905 read 0x41986ec thru 0x4199f46 (0x185b bytes) 2024-09-16T11:15:29.386 INFO:teuthology.orchestra.run.smithi162.stdout:5907 punch from 0x64d52fc to 0x64e0fac, (0xbcb0 bytes) 2024-09-16T11:15:29.386 INFO:teuthology.orchestra.run.smithi162.stdout:5908 read 0x87173e4 thru 0x871d3d2 (0x5fef bytes) 2024-09-16T11:15:29.386 INFO:teuthology.orchestra.run.smithi162.stdout:5909 read 0x4fa1d94 thru 0x4fa64a1 (0x470e bytes) 2024-09-16T11:15:29.387 INFO:teuthology.orchestra.run.smithi162.stdout:5910 read 0x81a220b thru 0x81ab1fe (0x8ff4 bytes) 2024-09-16T11:15:29.388 INFO:teuthology.orchestra.run.smithi162.stdout:5912 write 0x9e09f3 thru 0x9efc6d (0xf27b bytes) 2024-09-16T11:15:29.391 INFO:teuthology.orchestra.run.smithi162.stdout:5914 read 0x3cd7c5c thru 0x3ce6edf (0xf284 bytes) 2024-09-16T11:15:29.392 INFO:teuthology.orchestra.run.smithi162.stdout:5915 read 0x73f2093 thru 0x73f20f8 (0x66 bytes) 2024-09-16T11:15:29.407 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:29.405+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:29.414 INFO:teuthology.orchestra.run.smithi162.stdout:5916 trunc from 0xe54dde2 to 0x32f8272 2024-09-16T11:15:29.429 INFO:teuthology.orchestra.run.smithi162.stdout:5917 read 0x315a466 thru 0x315d650 (0x31eb bytes) 2024-09-16T11:15:29.429 INFO:teuthology.orchestra.run.smithi162.stdout:5919 read 0x1069981 thru 0x1070970 (0x6ff0 bytes) 2024-09-16T11:15:29.430 INFO:teuthology.orchestra.run.smithi162.stdout:5921 write 0x83e0ba thru 0x843bf7 (0x5b3e bytes) 2024-09-16T11:15:29.435 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:29.436 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:29.436 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:29.436 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:29.436 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:29.445 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:29.454 INFO:teuthology.orchestra.run.smithi162.stdout:5923 write 0x840df97 thru 0x84178a0 (0x990a bytes) 2024-09-16T11:15:29.457 INFO:teuthology.orchestra.run.smithi162.stdout:5924 writesame 0x5601012 thru 0x560ea6c data_size 0x3(0xda5b bytes) 2024-09-16T11:15:29.473 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.473 INFO:teuthology.orchestra.run.smithi162.stdout:5926 write 0x1f58179 thru 0x1f64a72 (0xc8fa bytes) 2024-09-16T11:15:29.477 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.477 INFO:teuthology.orchestra.run.smithi162.stdout:5933 read 0x3f90426 thru 0x3f97735 (0x7310 bytes) 2024-09-16T11:15:29.478 INFO:teuthology.orchestra.run.smithi162.stdout:5934 write 0x3f2f05e thru 0x3f31ac0 (0x2a63 bytes) 2024-09-16T11:15:29.481 INFO:teuthology.orchestra.run.smithi162.stdout:5935 punch from 0x254e02 to 0x2603b8, (0xb5b6 bytes) 2024-09-16T11:15:29.482 INFO:teuthology.orchestra.run.smithi162.stdout:5939 write 0x23f7ad6 thru 0x24061ff (0xe72a bytes) 2024-09-16T11:15:29.485 INFO:teuthology.orchestra.run.smithi162.stdout:5940 read 0x4372883 thru 0x437af6f (0x86ed bytes) 2024-09-16T11:15:29.486 INFO:teuthology.orchestra.run.smithi162.stdout:5941 punch from 0x628fd30 to 0x62996af, (0x997f bytes) 2024-09-16T11:15:29.486 INFO:teuthology.orchestra.run.smithi162.stdout:5942 write 0x6f96d45 thru 0x6f970aa (0x366 bytes) 2024-09-16T11:15:29.490 INFO:teuthology.orchestra.run.smithi162.stdout:5943 trunc from 0x84178a1 to 0x8c6f4b6 2024-09-16T11:15:29.503 INFO:teuthology.orchestra.run.smithi162.stdout:5945 write 0x859a727 thru 0x859c86a (0x2144 bytes) 2024-09-16T11:15:29.508 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:29.502+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:29.523 INFO:teuthology.orchestra.run.smithi162.stdout:5947 write 0xa1043f8 thru 0xa10b886 (0x748f bytes) 2024-09-16T11:15:29.527 INFO:teuthology.orchestra.run.smithi162.stdout:5949 read 0x16395ab thru 0x1641650 (0x80a6 bytes) 2024-09-16T11:15:29.527 INFO:teuthology.orchestra.run.smithi162.stdout:5953 read 0x54ab6f0 thru 0x54af433 (0x3d44 bytes) 2024-09-16T11:15:29.538 INFO:teuthology.orchestra.run.smithi162.stdout:5954 trunc from 0xa10b887 to 0xe317130 2024-09-16T11:15:29.546 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:29.546 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.468022140000002, 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.547 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:29.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:29.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:29.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467981378000005, 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.555 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.556 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.556 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.556 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.556 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:29.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.556 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:29.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:29.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:29.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:29.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:29.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:29.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.563 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.564 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.564 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:29.564 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467950579999993, 2024-09-16T11:15:29.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:29.565 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.565 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.565 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:29.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:29.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:29.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:29.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:29.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:29.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:29.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:29.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:29.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:29.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:29.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.572 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.572 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:29.572 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:29.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.572 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.573 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467931992000004, 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.576 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:29.577 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:29.578 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.579 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.580 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.581 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467913940000003, 2024-09-16T11:15:29.582 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.584 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:29.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.587 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467894013000006, 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.588 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:29.589 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:29.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:29.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.592 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.593 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467875174, 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.595 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:29.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:29.597 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:29.598 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467683225000002, 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.599 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.600 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.601 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.602 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:29.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.604 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.604 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:29.604 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:29.604 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.604 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.605 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:29.606 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:29.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.606 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.606 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.606 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.606 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.607 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.607 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:29.607 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.467663944999998, 2024-09-16T11:15:29.607 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:29.607 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.608 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.608 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.608 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:29.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.609 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.609 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.609 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.609 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:29.609 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.610 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.610 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:29.610 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.610 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.610 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.611 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:29.611 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:29.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.611 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.611 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:29.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.613 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.614 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464172376999997, 2024-09-16T11:15:29.615 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:29.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:29.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:29.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:29.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:29.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464162306000006, 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:29.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464134002999998, 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:29.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464120073999993, 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:29.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:29.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:29.638 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464111310999996, 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:29.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:29.643 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464087492000004, 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:29.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.648 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464056369999994, 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:29.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.464008630999999, 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 67.463986523000003, 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.659 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.054792159000002, 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.663 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:29.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:29.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:29.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:29.670 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 16.054746178999999, 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:29.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:29.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:29.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:29.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:29.678 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:29.679 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:29.681 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5955 read 0x4ca64e thru 0x4d57ef (0xb1a2 bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5956 punch from 0x6a0d5db to 0x6a10230, (0x2c55 bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5957 trunc from 0xe317130 to 0xe4774e0 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5960 writesame 0xd841767 thru 0xd84d736 data_size 0x84(0xbfd0 bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5961 write 0x38546d5 thru 0x38638cc (0xf1f8 bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5962 trunc from 0xe4774e0 to 0xa21c950 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5964 read 0x692b63e thru 0x6933392 (0x7d55 bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5965 read 0x774736e thru 0x774a67b (0x330e bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:5966 write 0x277a946 thru 0x2781cbd (0x7378 bytes) 2024-09-16T11:15:29.683 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5969 writesame 0x7c400f9 thru 0x7c457bf data_size 0x5(0x56c7 bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5970 writesame 0x7327a36 thru 0x732f300 data_size 0x11(0x78cb bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5977 read 0x7932a16 thru 0x793b691 (0x8c7c bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5978 write 0x7b5a973 thru 0x7b6623b (0xb8c9 bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5979 punch from 0x7224b20 to 0x722c446, (0x7926 bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5980 write 0xb96d91d thru 0xb971d4c (0x4430 bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5981 read 0x1867063 thru 0x186963e (0x25dc bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5983 trunc from 0xb971d4d to 0xb239ee6 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5985 writesame 0xb3d54f5 thru 0xb3e17aa data_size 0x1(0xc2b6 bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5986 read 0x3ea2492 thru 0x3eb0a01 (0xe570 bytes) 2024-09-16T11:15:29.684 INFO:teuthology.orchestra.run.smithi162.stdout:5988 write 0x69080ad thru 0x6908131 (0x85 bytes) 2024-09-16T11:15:29.685 INFO:teuthology.orchestra.run.smithi162.stdout:5989 trunc from 0xb3e17ab to 0xa7188ed 2024-09-16T11:15:29.685 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:29.674+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:29.713 INFO:teuthology.orchestra.run.smithi162.stdout:5991 write 0xd67f109 thru 0xd68c225 (0xd11d bytes) 2024-09-16T11:15:29.716 INFO:teuthology.orchestra.run.smithi162.stdout:5992 read 0x262b4c0 thru 0x262b6d2 (0x213 bytes) 2024-09-16T11:15:29.717 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.717 INFO:teuthology.orchestra.run.smithi162.stdout:5994 write 0x6d70901 thru 0x6d7bfcf (0xb6cf bytes) 2024-09-16T11:15:29.722 INFO:teuthology.orchestra.run.smithi162.stdout:5995 trunc from 0xd68c226 to 0xe5e25a7 2024-09-16T11:15:29.733 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.733 INFO:teuthology.orchestra.run.smithi162.stdout:5999 punch from 0xd589912 to 0xd5911d5, (0x78c3 bytes) 2024-09-16T11:15:29.734 INFO:teuthology.orchestra.run.smithi162.stdout:compare and write disabled 2024-09-16T11:15:29.759 INFO:teuthology.orchestra.run.smithi162.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-16T11:15:29.781 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:29.778+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:29.789 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:29.789 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:29.789 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:29.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:29.825 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:29.845 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:29.869 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:29.882 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:29.878+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:29.891 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:29.897 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:29.897 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:29.898 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:29.898 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:29.898 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:29.906 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:29.914 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:29.934 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:29.955 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:29.976 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:29.978 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:29.979 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:29.982 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:29.981+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:30.004 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.004 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.013 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:30.084 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:30.078+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:30.109 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.109 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.119 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:30.186 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:30.182+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:30.221 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.221 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.230 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:30.287 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:30.282+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:30.333 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.333 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.343 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:30.387 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:30.385+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:30.438 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.438 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.447 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:30.488 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:30.482+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:30.589 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:30.589+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:30.594 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.594 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.603 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:30.613 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:15:30.690 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:30.686+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:30.704 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.704 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.713 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:30.791 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:30.786+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:30.822 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.822 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.832 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:30.892 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:30.889+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:30.973 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:30.974 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:30.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:15:30.983 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:30.993 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:30.989+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:31.078 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.078 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.087 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:31.093 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:31.093+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:31.182 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.182 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:31.194 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:31.193+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:31.274 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:31.278 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:15:31.285 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.285 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.289 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-16T11:15:30.602143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1738093270,"num_objects":2746,"num_object_clones":554,"num_object_copies":5492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2743,"num_whiteouts":712,"num_read":241278,"num_read_kb":1411234,"num_write":25400,"num_write_kb":751791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4708,"num_bytes_recovered":520858214,"num_keys_recovered":609,"num_objects_omap":44,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19080,"ondisk_log_size":19080,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":3122960,"kb_used_data":1008172,"kb_used_omap":11,"kb_used_meta":2114676,"kb_avail":657479920,"statfs":{"total":676457349120,"available":673259438080,"internally_reserved":0,"allocated":1032368128,"data_stored":1112699412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11928,"internal_metadata":2165428584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2500556,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1913,"num_read_kb":2411,"num_write":772,"num_write_kb":3156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":36983457,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.615363"},"pg_stats":[{"pgid":"4.1f","version":"141'365","reported_seq":5899,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.447471+0000","last_change":"2024-09-16T11:14:58.670830+0000","last_active":"2024-09-16T11:15:27.447471+0000","last_peered":"2024-09-16T11:15:27.447471+0000","last_clean":"2024-09-16T11:15:27.447471+0000","last_became_active":"2024-09-16T11:14:58.669893+0000","last_became_peered":"2024-09-16T11:14:58.669893+0000","last_unstale":"2024-09-16T11:15:27.447471+0000","last_undegraded":"2024-09-16T11:15:27.447471+0000","last_fullsized":"2024-09-16T11:15:27.447471+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":365,"log_dups_size":0,"ondisk_log_size":365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:03.555919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002295123,"stat_sum":{"num_bytes":32440281,"num_objects":94,"num_object_clones":23,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":26,"num_read":6300,"num_read_kb":31198,"num_write":343,"num_write_kb":11391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":8822592,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"139'680","reported_seq":10552,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.446378+0000","last_change":"2024-09-16T11:13:15.329907+0000","last_active":"2024-09-16T11:15:26.446378+0000","last_peered":"2024-09-16T11:15:26.446378+0000","last_clean":"2024-09-16T11:15:26.446378+0000","last_became_active":"2024-09-16T11:13:13.129077+0000","last_became_peered":"2024-09-16T11:13:13.129077+0000","last_unstale":"2024-09-16T11:15:26.446378+0000","last_undegraded":"2024-09-16T11:15:26.446378+0000","last_fullsized":"2024-09-16T11:15:26.446378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:56.981990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050881799999999997,"stat_sum":{"num_bytes":66229668,"num_objects":76,"num_object_clones":11,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":21,"num_read":12400,"num_read_kb":91410,"num_write":872,"num_write_kb":42861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"139'414","reported_seq":4450,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.438204+0000","last_change":"2024-09-16T11:13:15.119814+0000","last_active":"2024-09-16T11:15:26.438204+0000","last_peered":"2024-09-16T11:15:26.438204+0000","last_clean":"2024-09-16T11:15:26.438204+0000","last_became_active":"2024-09-16T11:13:13.110835+0000","last_became_peered":"2024-09-16T11:13:13.110835+0000","last_unstale":"2024-09-16T11:15:26.438204+0000","last_undegraded":"2024-09-16T11:15:26.438204+0000","last_fullsized":"2024-09-16T11:15:26.438204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:25.687407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063997100000000003,"stat_sum":{"num_bytes":46230289,"num_objects":82,"num_object_clones":18,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":19,"num_read":4677,"num_read_kb":26057,"num_write":546,"num_write_kb":14493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"141'892","reported_seq":8147,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.440470+0000","last_change":"2024-09-16T11:14:24.176887+0000","last_active":"2024-09-16T11:15:27.440470+0000","last_peered":"2024-09-16T11:15:27.440470+0000","last_clean":"2024-09-16T11:15:27.440470+0000","last_became_active":"2024-09-16T11:14:24.105273+0000","last_became_peered":"2024-09-16T11:14:24.105273+0000","last_unstale":"2024-09-16T11:15:27.440470+0000","last_undegraded":"2024-09-16T11:15:27.440470+0000","last_fullsized":"2024-09-16T11:15:27.440470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:44:10.103253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038046360000000001,"stat_sum":{"num_bytes":41474335,"num_objects":93,"num_object_clones":19,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":26,"num_read":10975,"num_read_kb":37510,"num_write":1283,"num_write_kb":18176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2431957,"num_keys_recovered":68,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"141'424","reported_seq":7146,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.443329+0000","last_change":"2024-09-16T11:15:07.454596+0000","last_active":"2024-09-16T11:15:27.443329+0000","last_peered":"2024-09-16T11:15:27.443329+0000","last_clean":"2024-09-16T11:15:27.443329+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:15:27.443329+0000","last_undegraded":"2024-09-16T11:15:27.443329+0000","last_fullsized":"2024-09-16T11:15:27.443329+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029419009999999998,"stat_sum":{"num_bytes":20286751,"num_objects":75,"num_object_clones":12,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":18,"num_read":7224,"num_read_kb":24218,"num_write":562,"num_write_kb":14641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"141'564","reported_seq":7295,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.514223+0000","last_change":"2024-09-16T11:15:05.439505+0000","last_active":"2024-09-16T11:15:27.514223+0000","last_peered":"2024-09-16T11:15:27.514223+0000","last_clean":"2024-09-16T11:15:27.514223+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:15:27.514223+0000","last_undegraded":"2024-09-16T11:15:27.514223+0000","last_fullsized":"2024-09-16T11:15:27.514223+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024433810000000001,"stat_sum":{"num_bytes":45458930,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":18,"num_read":7922,"num_read_kb":40449,"num_write":742,"num_write_kb":21981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"141'672","reported_seq":2734,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.485302+0000","last_change":"2024-09-16T11:14:23.090655+0000","last_active":"2024-09-16T11:15:27.485302+0000","last_peered":"2024-09-16T11:15:27.485302+0000","last_clean":"2024-09-16T11:15:27.485302+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:15:27.485302+0000","last_undegraded":"2024-09-16T11:15:27.485302+0000","last_fullsized":"2024-09-16T11:15:27.485302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072097299999999997,"stat_sum":{"num_bytes":54776275,"num_objects":100,"num_object_clones":23,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":29,"num_read":4451,"num_read_kb":20695,"num_write":1032,"num_write_kb":18669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"139'620","reported_seq":4847,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.440146+0000","last_change":"2024-09-16T11:15:08.474104+0000","last_active":"2024-09-16T11:15:26.440146+0000","last_peered":"2024-09-16T11:15:26.440146+0000","last_clean":"2024-09-16T11:15:26.440146+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:15:26.440146+0000","last_undegraded":"2024-09-16T11:15:26.440146+0000","last_fullsized":"2024-09-16T11:15:26.440146+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055416799999999996,"stat_sum":{"num_bytes":84121943,"num_objects":81,"num_object_clones":13,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":17,"num_read":6285,"num_read_kb":35627,"num_write":712,"num_write_kb":29154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"141'1480","reported_seq":6114,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.446640+0000","last_change":"2024-09-16T11:13:15.120697+0000","last_active":"2024-09-16T11:15:27.446640+0000","last_peered":"2024-09-16T11:15:27.446640+0000","last_clean":"2024-09-16T11:15:27.446640+0000","last_became_active":"2024-09-16T11:13:13.131852+0000","last_became_peered":"2024-09-16T11:13:13.131852+0000","last_unstale":"2024-09-16T11:15:27.446640+0000","last_undegraded":"2024-09-16T11:15:27.446640+0000","last_fullsized":"2024-09-16T11:15:27.446640+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1480,"log_dups_size":0,"ondisk_log_size":1480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:23.813033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041000900000000002,"stat_sum":{"num_bytes":26856235,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":13,"num_read":11635,"num_read_kb":16377,"num_write":2580,"num_write_kb":8656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"139'376","reported_seq":7051,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.446457+0000","last_change":"2024-09-16T11:13:15.330097+0000","last_active":"2024-09-16T11:15:26.446457+0000","last_peered":"2024-09-16T11:15:26.446457+0000","last_clean":"2024-09-16T11:15:26.446457+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:15:26.446457+0000","last_undegraded":"2024-09-16T11:15:26.446457+0000","last_fullsized":"2024-09-16T11:15:26.446457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061357700000000002,"stat_sum":{"num_bytes":59976669,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":22,"num_read":7444,"num_read_kb":49605,"num_write":448,"num_write_kb":16180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"139'378","reported_seq":3484,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.578641+0000","last_change":"2024-09-16T11:14:58.670333+0000","last_active":"2024-09-16T11:15:26.578641+0000","last_peered":"2024-09-16T11:15:26.578641+0000","last_clean":"2024-09-16T11:15:26.578641+0000","last_became_active":"2024-09-16T11:14:58.668312+0000","last_became_peered":"2024-09-16T11:14:58.668312+0000","last_unstale":"2024-09-16T11:15:26.578641+0000","last_undegraded":"2024-09-16T11:15:26.578641+0000","last_fullsized":"2024-09-16T11:15:26.578641+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:09:03.441602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065955699999999996,"stat_sum":{"num_bytes":48094730,"num_objects":88,"num_object_clones":18,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":30,"num_read":3242,"num_read_kb":16405,"num_write":353,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":9395442,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"141'545","reported_seq":6245,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.446557+0000","last_change":"2024-09-16T11:15:26.448523+0000","last_active":"2024-09-16T11:15:27.446557+0000","last_peered":"2024-09-16T11:15:27.446557+0000","last_clean":"2024-09-16T11:15:27.446557+0000","last_became_active":"2024-09-16T11:15:26.444873+0000","last_became_peered":"2024-09-16T11:15:26.444873+0000","last_unstale":"2024-09-16T11:15:27.446557+0000","last_undegraded":"2024-09-16T11:15:27.446557+0000","last_fullsized":"2024-09-16T11:15:27.446557+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:25.029909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066315600000000003,"stat_sum":{"num_bytes":81865169,"num_objects":87,"num_object_clones":22,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":26,"num_read":6954,"num_read_kb":59596,"num_write":523,"num_write_kb":27707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":17072176,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"138'446","reported_seq":3965,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.440784+0000","last_change":"2024-09-16T11:15:05.443267+0000","last_active":"2024-09-16T11:15:26.440784+0000","last_peered":"2024-09-16T11:15:26.440784+0000","last_clean":"2024-09-16T11:15:26.440784+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:15:26.440784+0000","last_undegraded":"2024-09-16T11:15:26.440784+0000","last_fullsized":"2024-09-16T11:15:26.440784+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028235669999999999,"stat_sum":{"num_bytes":46707291,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":16,"num_read":4462,"num_read_kb":38147,"num_write":485,"num_write_kb":23390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"141'413","reported_seq":5169,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.443004+0000","last_change":"2024-09-16T11:15:26.451326+0000","last_active":"2024-09-16T11:15:27.443004+0000","last_peered":"2024-09-16T11:15:27.443004+0000","last_clean":"2024-09-16T11:15:27.443004+0000","last_became_active":"2024-09-16T11:15:26.449417+0000","last_became_peered":"2024-09-16T11:15:26.449417+0000","last_unstale":"2024-09-16T11:15:27.443004+0000","last_undegraded":"2024-09-16T11:15:27.443004+0000","last_fullsized":"2024-09-16T11:15:27.443004+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.160547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001145994,"stat_sum":{"num_bytes":85840409,"num_objects":88,"num_object_clones":18,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":19,"num_read":4566,"num_read_kb":52983,"num_write":661,"num_write_kb":23815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":31728710,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"139'556","reported_seq":6730,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.437806+0000","last_change":"2024-09-16T11:13:15.119821+0000","last_active":"2024-09-16T11:15:26.437806+0000","last_peered":"2024-09-16T11:15:26.437806+0000","last_clean":"2024-09-16T11:15:26.437806+0000","last_became_active":"2024-09-16T11:13:13.131936+0000","last_became_peered":"2024-09-16T11:13:13.131936+0000","last_unstale":"2024-09-16T11:15:26.437806+0000","last_undegraded":"2024-09-16T11:15:26.437806+0000","last_fullsized":"2024-09-16T11:15:26.437806+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:24.053000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059135899999999996,"stat_sum":{"num_bytes":102120377,"num_objects":96,"num_object_clones":20,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":30,"num_read":7907,"num_read_kb":96148,"num_write":626,"num_write_kb":44844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"138'507","reported_seq":6645,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.437630+0000","last_change":"2024-09-16T11:14:45.570341+0000","last_active":"2024-09-16T11:15:26.437630+0000","last_peered":"2024-09-16T11:15:26.437630+0000","last_clean":"2024-09-16T11:15:26.437630+0000","last_became_active":"2024-09-16T11:14:45.569559+0000","last_became_peered":"2024-09-16T11:14:45.569559+0000","last_unstale":"2024-09-16T11:15:26.437630+0000","last_undegraded":"2024-09-16T11:15:26.437630+0000","last_fullsized":"2024-09-16T11:15:26.437630+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:51.205615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050522900000000005,"stat_sum":{"num_bytes":50414091,"num_objects":84,"num_object_clones":16,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":24,"num_read":7334,"num_read_kb":37445,"num_write":691,"num_write_kb":22993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":18357825,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":134,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.446510+0000","last_change":"2024-09-16T11:12:56.908744+0000","last_active":"2024-09-16T11:15:26.446510+0000","last_peered":"2024-09-16T11:15:26.446510+0000","last_clean":"2024-09-16T11:15:26.446510+0000","last_became_active":"2024-09-16T11:12:54.829933+0000","last_became_peered":"2024-09-16T11:12:54.829933+0000","last_unstale":"2024-09-16T11:15:26.446510+0000","last_undegraded":"2024-09-16T11:15:26.446510+0000","last_fullsized":"2024-09-16T11:15:26.446510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:04:13.637362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"141'796","reported_seq":4950,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.507257+0000","last_change":"2024-09-16T11:15:08.474134+0000","last_active":"2024-09-16T11:15:27.507257+0000","last_peered":"2024-09-16T11:15:27.507257+0000","last_clean":"2024-09-16T11:15:27.507257+0000","last_became_active":"2024-09-16T11:15:08.472016+0000","last_became_peered":"2024-09-16T11:15:08.472016+0000","last_unstale":"2024-09-16T11:15:27.507257+0000","last_undegraded":"2024-09-16T11:15:27.507257+0000","last_fullsized":"2024-09-16T11:15:27.507257+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:25.422198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071029199999999995,"stat_sum":{"num_bytes":35925821,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":28,"num_read":7146,"num_read_kb":17292,"num_write":1005,"num_write_kb":8512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":12894750,"num_keys_recovered":93,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.443160+0000","last_change":"2024-09-16T11:12:56.845988+0000","last_active":"2024-09-16T11:15:26.443160+0000","last_peered":"2024-09-16T11:15:26.443160+0000","last_clean":"2024-09-16T11:15:26.443160+0000","last_became_active":"2024-09-16T11:12:54.832631+0000","last_became_peered":"2024-09-16T11:12:54.832631+0000","last_unstale":"2024-09-16T11:15:26.443160+0000","last_undegraded":"2024-09-16T11:15:26.443160+0000","last_fullsized":"2024-09-16T11:15:26.443160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:09.042057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074104499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"138'335","reported_seq":4458,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.437762+0000","last_change":"2024-09-16T11:13:15.119575+0000","last_active":"2024-09-16T11:15:26.437762+0000","last_peered":"2024-09-16T11:15:26.437762+0000","last_clean":"2024-09-16T11:15:26.437762+0000","last_became_active":"2024-09-16T11:13:13.131951+0000","last_became_peered":"2024-09-16T11:13:13.131951+0000","last_unstale":"2024-09-16T11:15:26.437762+0000","last_undegraded":"2024-09-16T11:15:26.437762+0000","last_fullsized":"2024-09-16T11:15:26.437762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:26.039221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051524000000000003,"stat_sum":{"num_bytes":48788005,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":28,"num_read":4421,"num_read_kb":35881,"num_write":350,"num_write_kb":11484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.448151+0000","last_change":"2024-09-16T11:12:57.281972+0000","last_active":"2024-09-16T11:15:26.448151+0000","last_peered":"2024-09-16T11:15:26.448151+0000","last_clean":"2024-09-16T11:15:26.448151+0000","last_became_active":"2024-09-16T11:12:55.084251+0000","last_became_peered":"2024-09-16T11:12:55.084251+0000","last_unstale":"2024-09-16T11:15:26.448151+0000","last_undegraded":"2024-09-16T11:15:26.448151+0000","last_fullsized":"2024-09-16T11:15:26.448151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:06:23.985377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073665700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"141'847","reported_seq":16450,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.445895+0000","last_change":"2024-09-16T11:14:58.670274+0000","last_active":"2024-09-16T11:15:27.445895+0000","last_peered":"2024-09-16T11:15:27.445895+0000","last_clean":"2024-09-16T11:15:27.445895+0000","last_became_active":"2024-09-16T11:14:58.669120+0000","last_became_peered":"2024-09-16T11:14:58.669120+0000","last_unstale":"2024-09-16T11:15:27.445895+0000","last_undegraded":"2024-09-16T11:15:27.445895+0000","last_fullsized":"2024-09-16T11:15:27.445895+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:13.376287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075674599999999998,"stat_sum":{"num_bytes":45678758,"num_objects":103,"num_object_clones":23,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":27,"num_read":18402,"num_read_kb":46297,"num_write":1156,"num_write_kb":20660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":9456611,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.446778+0000","last_change":"2024-09-16T11:13:52.391053+0000","last_active":"2024-09-16T11:15:26.446778+0000","last_peered":"2024-09-16T11:15:26.446778+0000","last_clean":"2024-09-16T11:15:26.446778+0000","last_became_active":"2024-09-16T11:13:52.390767+0000","last_became_peered":"2024-09-16T11:13:52.390767+0000","last_unstale":"2024-09-16T11:15:26.446778+0000","last_undegraded":"2024-09-16T11:15:26.446778+0000","last_fullsized":"2024-09-16T11:15:26.446778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:14.009852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"141'406","reported_seq":7835,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.443137+0000","last_change":"2024-09-16T11:15:26.449429+0000","last_active":"2024-09-16T11:15:27.443137+0000","last_peered":"2024-09-16T11:15:27.443137+0000","last_clean":"2024-09-16T11:15:27.443137+0000","last_became_active":"2024-09-16T11:15:26.444117+0000","last_became_peered":"2024-09-16T11:15:26.444117+0000","last_unstale":"2024-09-16T11:15:27.443137+0000","last_undegraded":"2024-09-16T11:15:27.443137+0000","last_fullsized":"2024-09-16T11:15:27.443137+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:50:59.504481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815220000000001,"stat_sum":{"num_bytes":25176905,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":21,"num_read":7422,"num_read_kb":28645,"num_write":511,"num_write_kb":10788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":37067586,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.440670+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:15:26.440670+0000","last_peered":"2024-09-16T11:15:26.440670+0000","last_clean":"2024-09-16T11:15:26.440670+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:15:26.440670+0000","last_undegraded":"2024-09-16T11:15:26.440670+0000","last_fullsized":"2024-09-16T11:15:26.440670+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":60,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.442730+0000","last_change":"2024-09-16T11:14:55.442877+0000","last_active":"2024-09-16T11:15:26.442730+0000","last_peered":"2024-09-16T11:15:26.442730+0000","last_clean":"2024-09-16T11:15:26.442730+0000","last_became_active":"2024-09-16T11:14:55.438669+0000","last_became_peered":"2024-09-16T11:14:55.438669+0000","last_unstale":"2024-09-16T11:15:26.442730+0000","last_undegraded":"2024-09-16T11:15:26.442730+0000","last_fullsized":"2024-09-16T11:15:26.442730+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:42:40.469472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"138'534","reported_seq":6150,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.525142+0000","last_change":"2024-09-16T11:14:23.105967+0000","last_active":"2024-09-16T11:15:26.525142+0000","last_peered":"2024-09-16T11:15:26.525142+0000","last_clean":"2024-09-16T11:15:26.525142+0000","last_became_active":"2024-09-16T11:14:23.092519+0000","last_became_peered":"2024-09-16T11:14:23.092519+0000","last_unstale":"2024-09-16T11:15:26.525142+0000","last_undegraded":"2024-09-16T11:15:26.525142+0000","last_fullsized":"2024-09-16T11:15:26.525142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:58.815065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001093002,"stat_sum":{"num_bytes":65523938,"num_objects":76,"num_object_clones":14,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":14,"num_read":7273,"num_read_kb":59428,"num_write":904,"num_write_kb":29264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":14007837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":72,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.524930+0000","last_change":"2024-09-16T11:14:20.060782+0000","last_active":"2024-09-16T11:15:26.524930+0000","last_peered":"2024-09-16T11:15:26.524930+0000","last_clean":"2024-09-16T11:15:26.524930+0000","last_became_active":"2024-09-16T11:14:20.060295+0000","last_became_peered":"2024-09-16T11:14:20.060295+0000","last_unstale":"2024-09-16T11:15:26.524930+0000","last_undegraded":"2024-09-16T11:15:26.524930+0000","last_fullsized":"2024-09-16T11:15:26.524930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:30.845736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.525117+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:15:26.525117+0000","last_peered":"2024-09-16T11:15:26.525117+0000","last_clean":"2024-09-16T11:15:26.525117+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:15:26.525117+0000","last_undegraded":"2024-09-16T11:15:26.525117+0000","last_fullsized":"2024-09-16T11:15:26.525117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"141'552","reported_seq":3156,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.514167+0000","last_change":"2024-09-16T11:15:27.461872+0000","last_active":"2024-09-16T11:15:27.514167+0000","last_peered":"2024-09-16T11:15:27.514167+0000","last_clean":"2024-09-16T11:15:27.514167+0000","last_became_active":"2024-09-16T11:15:27.446728+0000","last_became_peered":"2024-09-16T11:15:27.446728+0000","last_unstale":"2024-09-16T11:15:27.514167+0000","last_undegraded":"2024-09-16T11:15:27.514167+0000","last_fullsized":"2024-09-16T11:15:27.514167+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:58.904831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014276930000000001,"stat_sum":{"num_bytes":19357915,"num_objects":89,"num_object_clones":22,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":23,"num_read":3531,"num_read_kb":9559,"num_write":687,"num_write_kb":17051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":7507710,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"51'1","reported_seq":151,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.444513+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:15:26.444513+0000","last_peered":"2024-09-16T11:15:26.444513+0000","last_clean":"2024-09-16T11:15:26.444513+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:15:26.444513+0000","last_undegraded":"2024-09-16T11:15:26.444513+0000","last_fullsized":"2024-09-16T11:15:26.444513+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.447637+0000","last_change":"2024-09-16T11:12:57.281743+0000","last_active":"2024-09-16T11:15:26.447637+0000","last_peered":"2024-09-16T11:15:26.447637+0000","last_clean":"2024-09-16T11:15:26.447637+0000","last_became_active":"2024-09-16T11:12:54.828272+0000","last_became_peered":"2024-09-16T11:12:54.828272+0000","last_unstale":"2024-09-16T11:15:26.447637+0000","last_undegraded":"2024-09-16T11:15:26.447637+0000","last_fullsized":"2024-09-16T11:15:26.447637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:27.558486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068397200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"141'406","reported_seq":3190,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.446143+0000","last_change":"2024-09-16T11:13:15.331158+0000","last_active":"2024-09-16T11:15:27.446143+0000","last_peered":"2024-09-16T11:15:27.446143+0000","last_clean":"2024-09-16T11:15:27.446143+0000","last_became_active":"2024-09-16T11:13:13.110085+0000","last_became_peered":"2024-09-16T11:13:13.110085+0000","last_unstale":"2024-09-16T11:15:27.446143+0000","last_undegraded":"2024-09-16T11:15:27.446143+0000","last_fullsized":"2024-09-16T11:15:27.446143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:43.994646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001220874,"stat_sum":{"num_bytes":57612936,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":19,"num_read":3271,"num_read_kb":98548,"num_write":447,"num_write_kb":53237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"96'2","reported_seq":141,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.438101+0000","last_change":"2024-09-16T11:14:55.446597+0000","last_active":"2024-09-16T11:15:26.438101+0000","last_peered":"2024-09-16T11:15:26.438101+0000","last_clean":"2024-09-16T11:15:26.438101+0000","last_became_active":"2024-09-16T11:14:55.438591+0000","last_became_peered":"2024-09-16T11:14:55.438591+0000","last_unstale":"2024-09-16T11:15:26.438101+0000","last_undegraded":"2024-09-16T11:15:26.438101+0000","last_fullsized":"2024-09-16T11:15:26.438101+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:11:47.479160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":133,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.525104+0000","last_change":"2024-09-16T11:14:55.442911+0000","last_active":"2024-09-16T11:15:26.525104+0000","last_peered":"2024-09-16T11:15:26.525104+0000","last_clean":"2024-09-16T11:15:26.525104+0000","last_became_active":"2024-09-16T11:14:55.440414+0000","last_became_peered":"2024-09-16T11:14:55.440414+0000","last_unstale":"2024-09-16T11:15:26.525104+0000","last_undegraded":"2024-09-16T11:15:26.525104+0000","last_fullsized":"2024-09-16T11:15:26.525104+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:20.765225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":222,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.447725+0000","last_change":"2024-09-16T11:12:49.907258+0000","last_active":"2024-09-16T11:15:26.447725+0000","last_peered":"2024-09-16T11:15:26.447725+0000","last_clean":"2024-09-16T11:15:26.447725+0000","last_became_active":"2024-09-16T11:12:49.905012+0000","last_became_peered":"2024-09-16T11:12:49.905012+0000","last_unstale":"2024-09-16T11:15:26.447725+0000","last_undegraded":"2024-09-16T11:15:26.447725+0000","last_fullsized":"2024-09-16T11:15:26.447725+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:14:42.267579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"138'434","reported_seq":5830,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.446452+0000","last_change":"2024-09-16T11:15:06.451311+0000","last_active":"2024-09-16T11:15:26.446452+0000","last_peered":"2024-09-16T11:15:26.446452+0000","last_clean":"2024-09-16T11:15:26.446452+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:15:26.446452+0000","last_undegraded":"2024-09-16T11:15:26.446452+0000","last_fullsized":"2024-09-16T11:15:26.446452+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059007199999999997,"stat_sum":{"num_bytes":36512219,"num_objects":91,"num_object_clones":20,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":14,"num_read":5992,"num_read_kb":32223,"num_write":474,"num_write_kb":15250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"141'766","reported_seq":2415,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.447870+0000","last_change":"2024-09-16T11:14:58.670074+0000","last_active":"2024-09-16T11:15:27.447870+0000","last_peered":"2024-09-16T11:15:27.447870+0000","last_clean":"2024-09-16T11:15:27.447870+0000","last_became_active":"2024-09-16T11:14:58.666316+0000","last_became_peered":"2024-09-16T11:14:58.666316+0000","last_unstale":"2024-09-16T11:15:27.447870+0000","last_undegraded":"2024-09-16T11:15:27.447870+0000","last_fullsized":"2024-09-16T11:15:27.447870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:19.352762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081336500000000003,"stat_sum":{"num_bytes":32998042,"num_objects":99,"num_object_clones":21,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":25,"num_read":3666,"num_read_kb":23300,"num_write":1084,"num_write_kb":12043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":8381723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"141'575","reported_seq":9439,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.488439+0000","last_change":"2024-09-16T11:15:06.450843+0000","last_active":"2024-09-16T11:15:27.488439+0000","last_peered":"2024-09-16T11:15:27.488439+0000","last_clean":"2024-09-16T11:15:27.488439+0000","last_became_active":"2024-09-16T11:15:06.443397+0000","last_became_peered":"2024-09-16T11:15:06.443397+0000","last_unstale":"2024-09-16T11:15:27.488439+0000","last_undegraded":"2024-09-16T11:15:27.488439+0000","last_fullsized":"2024-09-16T11:15:27.488439+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:45.235919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066151299999999997,"stat_sum":{"num_bytes":137875432,"num_objects":92,"num_object_clones":19,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":19,"num_read":10333,"num_read_kb":86034,"num_write":694,"num_write_kb":27482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":13213368,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"139'456","reported_seq":3923,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.446720+0000","last_change":"2024-09-16T11:13:15.134244+0000","last_active":"2024-09-16T11:15:26.446720+0000","last_peered":"2024-09-16T11:15:26.446720+0000","last_clean":"2024-09-16T11:15:26.446720+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:26.446720+0000","last_undegraded":"2024-09-16T11:15:26.446720+0000","last_fullsized":"2024-09-16T11:15:26.446720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:23.499921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054346600000000004,"stat_sum":{"num_bytes":49758980,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":20,"num_read":3727,"num_read_kb":45513,"num_write":518,"num_write_kb":19077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"141'533","reported_seq":6769,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.539351+0000","last_change":"2024-09-16T11:13:15.329809+0000","last_active":"2024-09-16T11:15:27.539351+0000","last_peered":"2024-09-16T11:15:27.539351+0000","last_clean":"2024-09-16T11:15:27.539351+0000","last_became_active":"2024-09-16T11:13:13.110747+0000","last_became_peered":"2024-09-16T11:13:13.110747+0000","last_unstale":"2024-09-16T11:15:27.539351+0000","last_undegraded":"2024-09-16T11:15:27.539351+0000","last_fullsized":"2024-09-16T11:15:27.539351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:36:36.983993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047034000000000003,"stat_sum":{"num_bytes":55835801,"num_objects":85,"num_object_clones":17,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":26,"num_read":7994,"num_read_kb":26752,"num_write":562,"num_write_kb":21874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"141'1087","reported_seq":9943,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T11:15:27.522966+0000","last_change":"2024-09-16T11:15:04.705143+0000","last_active":"2024-09-16T11:15:27.522966+0000","last_peered":"2024-09-16T11:15:27.522966+0000","last_clean":"2024-09-16T11:15:27.522966+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:15:27.522966+0000","last_undegraded":"2024-09-16T11:15:27.522966+0000","last_fullsized":"2024-09-16T11:15:27.522966+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023754140000000002,"stat_sum":{"num_bytes":32783803,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":23,"num_read":12739,"num_read_kb":40984,"num_write":1766,"num_write_kb":19725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"138'897","reported_seq":16899,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.440660+0000","last_change":"2024-09-16T11:15:09.477208+0000","last_active":"2024-09-16T11:15:26.440660+0000","last_peered":"2024-09-16T11:15:26.440660+0000","last_clean":"2024-09-16T11:15:26.440660+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:15:26.440660+0000","last_undegraded":"2024-09-16T11:15:26.440660+0000","last_fullsized":"2024-09-16T11:15:26.440660+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025819319999999999,"stat_sum":{"num_bytes":66109012,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":27,"num_read":17009,"num_read_kb":66145,"num_write":1472,"num_write_kb":25038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"139'619","reported_seq":8239,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.440482+0000","last_change":"2024-09-16T11:15:07.455688+0000","last_active":"2024-09-16T11:15:26.440482+0000","last_peered":"2024-09-16T11:15:26.440482+0000","last_clean":"2024-09-16T11:15:26.440482+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:15:26.440482+0000","last_undegraded":"2024-09-16T11:15:26.440482+0000","last_fullsized":"2024-09-16T11:15:26.440482+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00256719,"stat_sum":{"num_bytes":71233584,"num_objects":83,"num_object_clones":20,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":19,"num_read":9485,"num_read_kb":96903,"num_write":732,"num_write_kb":65950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"138'427","reported_seq":4444,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T11:15:26.525518+0000","last_change":"2024-09-16T11:14:44.551857+0000","last_active":"2024-09-16T11:15:26.525518+0000","last_peered":"2024-09-16T11:15:26.525518+0000","last_clean":"2024-09-16T11:15:26.525518+0000","last_became_active":"2024-09-16T11:14:44.547942+0000","last_became_peered":"2024-09-16T11:14:44.547942+0000","last_unstale":"2024-09-16T11:15:26.525518+0000","last_undegraded":"2024-09-16T11:15:26.525518+0000","last_fullsized":"2024-09-16T11:15:26.525518+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:14:05.099400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001217035,"stat_sum":{"num_bytes":63438206,"num_objects":81,"num_object_clones":13,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":23,"num_read":5023,"num_read_kb":23805,"num_write":475,"num_write_kb":27383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":14969107,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1737502800,"num_objects":2739,"num_object_clones":554,"num_object_copies":5478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2739,"num_whiteouts":710,"num_read":241212,"num_read_kb":1411179,"num_write":25296,"num_write_kb":750160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4701,"num_bytes_recovered":520858112,"num_keys_recovered":609,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1068806144,"data_stored":1158580887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19002,"ondisk_log_size":19002,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422772,"kb_used_data":121248,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93949068,"statfs":{"total":96636764160,"available":96203845632,"internally_reserved":0,"allocated":124157952,"data_stored":127090584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.89300000000000002,"15min":0.89900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79000000000000004,"15min":0.79800000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1970000000000001,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.0589999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.877,"15min":0.90100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":0.96799999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98699999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.88900000000000001,"15min":0.90500000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2110000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.995,"15min":1.006},"min":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.0960000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.45700000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.371,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.216},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.97399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.502,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.54800000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.024,"5min":1.0629999999999999,"15min":1.069},"min":{"1min":0.56799999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.3220000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.125,"15min":1.131},"min":{"1min":0.60899999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.397},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1240000000000001,"15min":1.1339999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.93300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":449296,"kb_used_data":147772,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93922544,"statfs":{"total":96636764160,"available":96176685056,"internally_reserved":0,"allocated":151318528,"data_stored":164282768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2430000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.873,"5min":0.90900000000000003,"15min":0.91500000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.296,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.996}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.089,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.872,"15min":0.874},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.83399999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.083,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.997,"15min":1.0149999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.198,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.91200000000000003,"15min":0.90800000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.353,"5min":1.466,"15min":1.466},"last":0.79600000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.014},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.371,"5min":1.417,"15min":1.417},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.8759999999999999,"5min":1.22,"15min":1.1100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":15.597,"5min":15.597,"15min":15.597},"last":0.76500000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.91100000000000003,"15min":0.89900000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.498},{"interface":"front","average":{"1min":0.999,"5min":1.077,"15min":1.0900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5069999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.93300000000000005}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1000000000000001,"15min":1.0940000000000001},"min":{"1min":0.5,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.069,"5min":1.077,"15min":1.079},"min":{"1min":0.60999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.73999999999999999}]}]},{"osd":5,"up_from":0,"seq":55834574884,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492376,"kb_used_data":182544,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93879464,"statfs":{"total":96636764160,"available":96132571136,"internally_reserved":0,"allocated":186925056,"data_stored":200772885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0409999999999999,"5min":1.329,"15min":1.329},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.49399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.52600000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.88500000000000001,"15min":0.90700000000000003},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.149,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77900000000000003,"15min":0.78900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0129999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.90800000000000003,"15min":0.93000000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.3080000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.034,"15min":1.0580000000000001},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1890000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.879,"5min":1.0269999999999999,"15min":1.0509999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.2250000000000001,"5min":2.077,"15min":2.077},"last":1.008}]},{"osd":5,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.085},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1200000000000001,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0660000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.97499999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.115,"15min":1.145},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3939999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.256},{"interface":"front","average":{"1min":1.101,"5min":1.143,"15min":1.1499999999999999},"min":{"1min":0.54900000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:14:53 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.0980000000000001,"15min":1.1359999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.173,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.242,"15min":1.282},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.496,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.3069999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478724,"kb_used_data":181372,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93893116,"statfs":{"total":96636764160,"available":96146550784,"internally_reserved":0,"allocated":185724928,"data_stored":199359186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.131}]},{"osd":1,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1200000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0609999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78200000000000003,"15min":0.78000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.155},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98799999999999999,"15min":0.997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.98199999999999998}]},{"osd":5,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.022,"15min":1.028},"min":{"1min":0.55500000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.371,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0369999999999999,"15min":1.03},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.1000000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.081,"15min":1.089},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0509999999999999,"15min":1.046},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":1.2529999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:14:50 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0069999999999999,"15min":0.997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.288}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412744,"kb_used_data":102904,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":93959096,"statfs":{"total":96636764160,"available":96214114304,"internally_reserved":0,"allocated":105373696,"data_stored":117572589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.27000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.34899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75800000000000001,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.28,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.1240000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.94199999999999995,"15min":0.95399999999999996},"min":{"1min":0.64400000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.496,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.238,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.995,"15min":1.0089999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.143,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0640000000000001,"15min":1.0660000000000001},"min":{"1min":0.751,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.5509999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.075},"min":{"1min":0.65200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.369,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.0329999999999999,"15min":1.0449999999999999},"min":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.302,"5min":1.744,"15min":1.744},"last":0.96999999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.119,"15min":1.127},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.589,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.71799999999999997,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.304,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463852,"kb_used_data":166492,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93907988,"statfs":{"total":96636764160,"available":96161779712,"internally_reserved":0,"allocated":170487808,"data_stored":188064761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.0249999999999999,"5min":1.877,"15min":1.877},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.82399999999999995,"15min":0.85199999999999998},"min":{"1min":0.40300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89600000000000002,"5min":1.98,"15min":1.98},"last":0.92800000000000005}]},{"osd":2,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.71099999999999997,"15min":0.72799999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.96899999999999997,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.85599999999999998,"15min":0.86699999999999999},"min":{"1min":0.45800000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.278,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78300000000000003,"15min":0.79700000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0780000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77000000000000002,"15min":0.78300000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.109,"5min":1.784,"15min":1.784},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.94299999999999995,"15min":0.96199999999999997},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.155,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.749},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.044},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.306,"5min":2.597,"15min":2.597},"last":0.88200000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.091,"15min":1.1140000000000001},"min":{"1min":0.56699999999999995,"5min":0.501,"15min":0.501},"max":{"1min":1.4359999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0109999999999999,"15min":1.032},"min":{"1min":0.51200000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.26,"5min":2.266,"15min":2.266},"last":0.79500000000000004}]},{"osd":6,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.117,"15min":1.141},"min":{"1min":0.70499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.3799999999999999,"5min":2.552,"15min":2.552},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0960000000000001,"15min":1.1220000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.337,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:14:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.103,"15min":1.111},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.4710000000000001,"5min":2.512,"15min":2.512},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.1519999999999999,"15min":1.1739999999999999},"min":{"1min":0.64300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.4790000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.01}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403196,"kb_used_data":105840,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93968644,"statfs":{"total":96636764160,"available":96223891456,"internally_reserved":0,"allocated":108380160,"data_stored":115556639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.24399999999999999,"5min":0.215,"15min":0.215},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65000000000000002,"15min":0.64600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.034,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.66400000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.495},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66900000000000004,"15min":0.66500000000000004},"min":{"1min":0.33800000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.56499999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.95799999999999996,"15min":0.98199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4019999999999999,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.41799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3029999999999999,"5min":2.129,"15min":2.129},"last":0.60899999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.54900000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4830000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.0980000000000001,"15min":1.1220000000000001},"min":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.3420000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.874}]},{"osd":6,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.026,"15min":1.026},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.397,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0129999999999999,"15min":1.0209999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.4670000000000001,"5min":1.841,"15min":1.841},"last":0.71099999999999997}]},{"osd":7,"last update":"Mon Sep 16 11:14:49 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.034,"15min":1.042},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3839999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.753},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.0509999999999999,"15min":1.0700000000000001},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3600000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.81899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106102784,"data_stored":114128079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":168759296,"data_stored":187198560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":103682048,"data_stored":116727224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":184000512,"data_stored":198485046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":185163776,"data_stored":199864197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":52248576,"data_stored":56168601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":149610496,"data_stored":163432505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":119238656,"data_stored":122576675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:15:31.292 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T11:15:31.293 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:31.293 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T11:15:31.293 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-16T11:15:31.293 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0 2024-09-16T11:15:31.294 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:31.295 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:31.293+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:31.396 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:31.390+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:31.437 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.438 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.447 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:31.497 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:31.494+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:31.536 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.536 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:31.598 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:31.594+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:31.626 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:15:31.699 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:31.697+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:31.699 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.699 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.709 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:31.800 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:31.797+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:31.807 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.807 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.818 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:31.900 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:31.894+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:31.913 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:31.914 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:31.923 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:31.993 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:32.002 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:32.001+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:32.022 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:32.022 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:32.032 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:32.102 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:32.098+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:32.123 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:32.126 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:32.126 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:32.135 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:32.147 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:32.166 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:32.192 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:32.204 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:32.198+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:32.215 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:32.241 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:32.245 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:32.245 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:32.245 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:32.245 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:32.254 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:32.261 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:32.283 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:32.286 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:32.286 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:32.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:32.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:32.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:32.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:32.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:32.287 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:32.305 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:32.302+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:32.354 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:32.354 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:32.354 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:32.354 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:32.354 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:32.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:32.406 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:32.402+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384192732000002, 2024-09-16T11:15:32.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:32.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384151970000005, 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:32.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:32.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:32.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384121171999993, 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:32.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.478 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384102584000004, 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:32.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.483 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384084532000003, 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:32.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:32.488 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384064605000006, 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.384045766, 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.495 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:32.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.383853817000002, 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:32.500 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:32.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:32.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.505 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.383834536999998, 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:32.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:32.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380342968999997, 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.511 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:32.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:32.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380332898000006, 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.516 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:32.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380304594999998, 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.521 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:32.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.526 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380290665999993, 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.527 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.530 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380281902999997, 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.532 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:32.533 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.534 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:32.535 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:32.536 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380258084000005, 2024-09-16T11:15:32.537 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.538 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:32.539 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.540 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.541 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.542 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380226961999995, 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.544 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.545 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:32.546 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.547 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380179222999999, 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.548 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.549 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.550 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:32.551 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.552 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 70.380157115000003, 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.553 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:32.554 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.555 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.556 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:32.557 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.970962750999998, 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.558 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:32.559 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.560 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.561 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.562 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.563 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.564 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.970916770999999, 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.565 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.566 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:32.567 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:32.568 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.569 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.570 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.571 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.572 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.572 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.572 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:32.573 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:32.574 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:32.570+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:32.631 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-16T11:15:32.674 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:32.673+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:32.693 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:32.693 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:32.693 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:32.693 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:32.703 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:32.775 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:32.773+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:32.802 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:32.802 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:32.802 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:32.802 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:32.802 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:32.813 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:32.876 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:32.873+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:32.957 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 132.09133206999999, 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:32.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:32.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923360421, 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:32.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92334886, 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.968 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:32.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923335658, 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:32.972 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92332321799999, 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:32.977 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:32.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923309707, 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:32.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:32.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923298792, 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:32.987 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:32.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.991 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92327287400001, 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:32.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:32.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:32.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:32.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92325742, 2024-09-16T11:15:32.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:32.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:33.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923247534, 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.002 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:33.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:33.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923237771, 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.007 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:33.011 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923227594, 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:33.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:33.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.017 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923211112, 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:33.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:33.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92319641500001, 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.023 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:33.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923177972, 2024-09-16T11:15:33.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:33.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:33.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.033 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92316102300001, 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:33.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:33.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:33.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.039 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92314181, 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.041 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:33.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.044 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.923124594, 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:33.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92310334, 2024-09-16T11:15:33.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:33.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:33.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:33.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:33.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:33.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:33.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 104.92308337199999, 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:33.056 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.057 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:33.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:33.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:33.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:33.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.063 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:33.065 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:33.062+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:33.166 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:33.162+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:33.169 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:33.169 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4.ad3e2632 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 0.40997570799999999, 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.410017876, 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:33.170 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:15:33.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-09-16T11:15:33.176 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 1 2024-09-16T11:15:33.177 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.179 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:33.267 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:33.262+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:33.281 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:33.282 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:33.282 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:33.282 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:33.282 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.291 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:33.369 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:33.362+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:33.391 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:33.391 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50305 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.193700808, 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12055690199999999, 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50305 2024-09-16T11:15:33.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.193835+0000", 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197603+0000", 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0037686640000000001 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197610+0000", 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7970000000000003e-06 2024-09-16T11:15:33.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197621+0000", 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0991e-05 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197627+0000", 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4069999999999998e-06 2024-09-16T11:15:33.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.313910+0000", 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11628233 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314028+0000", 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118319 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.314391+0000", 2024-09-16T11:15:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036339400000000001 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.189831177, 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.397 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:33.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.18981511, 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:33.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:33.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.18979871499999, 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.18978914100001, 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:33.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:33.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.189779605, 2024-09-16T11:15:33.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:33.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:33.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:33.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.491942722000005, 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:33.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:33.428 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.806107468, 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:33.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:33.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:33.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871588124999999, 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:33.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:33.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871507454, 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:33.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:33.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:33.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:33.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:33.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:33.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:33.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:33.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:33.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:33.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:33.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:33.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:33.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871386061999999, 2024-09-16T11:15:33.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:33.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:33.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871333868000001, 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871277135, 2024-09-16T11:15:33.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:33.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:33.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:33.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871229935999999, 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871179966, 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:33.476 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:33.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:33.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.871112852, 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:33.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:33.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:33.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:33.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.869137766000001, 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.488 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:33.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:33.493 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.869089281000001, 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.494 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:33.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.868972252999999, 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:33.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:33.505 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 27.868881078000001, 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.510 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.511 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.511 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.512 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:33.513 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:33.506+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:33.613 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:33.613+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:33.619 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:33.619 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:33.619 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.628 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:33.636 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:15:33.715 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:33.710+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:33.725 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:33.726 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:33.726 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:33.726 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:33.726 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:33.816 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:33.810+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 164.05366923899999, 2024-09-16T11:15:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.665743539, 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.66559220900001, 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.66554072299999, 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.665470575, 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:33.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:33.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:33.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:33.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.66529810500001, 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:33.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:33.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.665239783, 2024-09-16T11:15:33.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:33.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:33.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:33.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.66508766300001, 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.883 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.66490914400001, 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:33.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:33.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:33.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:33.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:33.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:33.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:33.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.897 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.664876171, 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:33.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:33.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:33.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.903 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.11268844599999, 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.904 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:33.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:33.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:33.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:33.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:33.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.911 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.912 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.912 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:33.912 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.11255964899999, 2024-09-16T11:15:33.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:33.912 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.912 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.913 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:33.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:33.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:33.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.11250130400001, 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:33.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:33.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:33.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.11244691499999, 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.932 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:33.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 163.112312608, 2024-09-16T11:15:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:33.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:33.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:33.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:33.944 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 162.33106992699999, 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:33.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.950 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 162.33083179900001, 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:33.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:33.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 76.942259354000001, 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:33.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:33.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:194870 4.8 4:13a4eb47:::rbd_data.11b55ebb7981.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 9.3811658439999999, 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015060858999999999, 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.961 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 194870 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:33.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448525+0000", 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.187e-06 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448539+0000", 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3705e-05 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448566+0000", 2024-09-16T11:15:33.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6937e-05 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448576+0000", 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0032e-05 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448682+0000", 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106353 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448818+0000", 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135308 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.452297+0000", 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034793049999999998 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463364+0000", 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011067184000000001 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463431+0000", 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6397999999999997e-05 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463518+0000", 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7220999999999997e-05 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463584+0000", 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229000000000006e-05 2024-09-16T11:15:33.966 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:195215 4.15 4:ab483f24:::rbd_data.11b55ebb7981.000000000000000a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 8.7433722439999997, 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015135688, 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:33.967 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 195215 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086318+0000", 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.598e-06 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086326+0000", 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0180000000000001e-06 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086347+0000", 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0222000000000001e-05 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086359+0000", 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2155e-05 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086451+0000", 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1815000000000001e-05 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086557+0000", 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010589 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.088907+0000", 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002350651 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101185+0000", 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01227816 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101257+0000", 2024-09-16T11:15:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1883000000000003e-05 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101359+0000", 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101766 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101453+0000", 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3529999999999994e-05 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:33.973 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:33.973 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:33.974 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:33.973+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:33.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:34.075 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:34.070+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:34.076 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:34.076 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:34.076 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:34.076 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:34.086 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:34.176 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:34.173+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:34.182 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:34.182 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:34.183 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:34.183 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:34.183 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:34.191 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:34.277 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:34.274+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:34.291 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:34.291 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.11443987, 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.110768361, 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.108703069, 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:34.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:34.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.10868921300001, 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:34.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.310 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.108675338, 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:34.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:34.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:34.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.108661571, 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:34.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:34.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.108614062, 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.10832824400001, 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:34.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.01566442799999, 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:34.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:34.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:34.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:34.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:34.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.01551524600001, 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:34.334 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:34.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:34.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.027114026, 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.340 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:34.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.344 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.027067066000001, 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:34.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:34.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.027048374, 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:34.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:34.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.353 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.027032174999999, 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:34.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:34.357 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.027017133000001, 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.358 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.362 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.027001383000002, 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:34.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:34.367 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.026986093000001, 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:34.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:34.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:34.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:34.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.026970514999999, 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.373 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.377 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.026955535999999, 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:34.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:34.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.5314657839999999, 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:15:34.382 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:15:34.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.389 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:34.389 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:34.391 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:34.392 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:34.389+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:34.492 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:34.489+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:34.537 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:34.537 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:34.537 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:34.538 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:34.547 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:34.594 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:34.593+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:34.639 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:34.639 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:34.640 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:34.651 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:34.657 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:15:34.694 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:34.690+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:34.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-16T11:15:34.738+0000 7f4ca4ddd700 0 --2- 172.21.15.162:0/2524401303 >> v2:172.21.15.162:3300/0 conn(0x7f4ca006b570 0x7f4ca006b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:34.750 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 87.253790018999993, 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:34.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:34.757 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.000377220999994, 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.758 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 73.000107904999993, 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:34.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.999709394999996, 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:34.767 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.772 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.999420607000005, 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:34.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:34.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:34.777 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.997721190999997, 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:34.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.997642874999997, 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.782 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:34.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:34.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.997609135999994, 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.997568391000001, 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.791 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.994367943, 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:34.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.994349885999995, 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:34.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.994323897000001, 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.805 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.993625465999997, 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:34.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.993612906999999, 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:34.814 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:34.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.083937153999997, 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.819 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:34.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:34.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.826 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.401511649, 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:34.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:34.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:34.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:34.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.833 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.398646756000002, 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:34.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:34.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.38960084, 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:34.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:34.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:34.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.332185334999998, 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:34.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:34.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:34.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:34.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 30.28592617, 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:34.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:34.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:34.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:34.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:34.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:34.860 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:34.861 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:34.862 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:34.863 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:34.863 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:34.863 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:34.863 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:34.863 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:34.858+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:34.868 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:34.890 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:34.910 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:34.932 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:34.935 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:34.966 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:34.966 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:34.966 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:34.962+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:34.976 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:35.064 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:35.058+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:35.119 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.119 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.128 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:35.165 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:35.161+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:35.226 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.226 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:35.266 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:35.265+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:35.339 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.339 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.349 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:35.367 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:35.362+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:35.453 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.453 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.465 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:35.467 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:35.462+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:35.568 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:35.565+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:35.573 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.573 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.584 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:35.669 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:35.662+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:35.684 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.684 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.692 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:35.770 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:35.766+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:35.789 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.789 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:35.871 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:35.866+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:35.898 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:35.899 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:35.907 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:35.972 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:35.966+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:36.003 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.003 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.012 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:36.075 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:36.069+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:36.114 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.114 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.123 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:36.173 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:36.173+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:36.268 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.268 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.274 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:36.273+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:36.376 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:36.374+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:36.420 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.421 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.429 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:36.476 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:36.473+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:36.530 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.530 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.539 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:36.582 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:36.581+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:36.672 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:15:36.678 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:36.676+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:36.688 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.689 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.698 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:36.779 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:36.776+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:36.793 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.793 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.803 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:36.879 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:36.876+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:36.946 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:36.946 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:36.947 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:36.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:36.980 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:36.976+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:37.054 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:37.055 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:37.063 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:37.086 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:37.106 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:37.106 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:37.104+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:37.126 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:37.146 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:37.160 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:37.160 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:37.160 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:37.161 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:37.168 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:37.169 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:37.182 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:37.180+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:37.189 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:37.224 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:37.258 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:37.261 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:37.261 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:37.261 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:37.261 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:37.262 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:37.282 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:37.278+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:37.291 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:37.292 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:37.292 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:37.292 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:37.292 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:37.300 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:37.384 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:37.380+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:37.446 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.26925396199999, 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.26558245299999, 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.263517161, 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.263503305, 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:37.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:37.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.26348943000001, 2024-09-16T11:15:37.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:37.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.469 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.26347566299999, 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.470 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:37.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.263428154, 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:37.474 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:37.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.263142336, 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:37.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.17047852, 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:37.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.170329338, 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:37.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:37.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:37.495 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181928118000002, 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:37.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:37.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181881158000003, 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:37.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:37.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:37.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181862465999998, 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:37.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:37.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181846266999997, 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:37.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:37.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181831225000003, 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:37.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:37.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:37.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181815475000001, 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181800185, 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:37.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181784606999997, 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:37.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.181769627999998, 2024-09-16T11:15:37.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:37.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.537 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.6862798760000004, 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:15:37.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:15:37.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:37.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.544 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.545 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.545 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:37.547 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:37.548 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:37.544+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:37.643 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 6, 7, 2, 4] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:37.643 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:37.644 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 3, 6, 7, 2, 4] 2024-09-16T11:15:37.644 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-16T11:15:37.647 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:37.644+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:37.653 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:37.653 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:37.653 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:37.653 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:37.666 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:37.667 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:15:37.747 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:37.744+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:37.764 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:37.764 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:37.765 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:37.775 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:37.848 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:37.842+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:37.881 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.384686693999996, 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:37.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:37.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:37.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.131273895999996, 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.890 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:37.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.131004579999995, 2024-09-16T11:15:37.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:37.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:37.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.130606069999999, 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.899 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.130317281999993, 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.903 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:37.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.907 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.128617865999999, 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:37.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:37.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.128539549999999, 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:37.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:37.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.916 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.128505810999997, 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.917 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:37.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.128465066000004, 2024-09-16T11:15:37.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:37.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:37.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.125264618000003, 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.926 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:37.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:37.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:37.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.125246560999997, 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.934 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:37.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:37.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.125220572000003, 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:37.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:37.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.124522141, 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:37.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.124509582000002, 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.947 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:37.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:37.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.214833829, 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.952 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:37.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:37.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.532408324000002, 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:37.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:37.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.529543431, 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:37.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:37.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.972 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.520497515000002, 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:37.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:37.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:37.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.463082010000001, 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.979 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:37.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:37.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.985 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 33.416822844999999, 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:37.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:37.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:37.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:37.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:37.992 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:37.994 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:37.994 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:15:37.950+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e150 definitely_dead 0 2024-09-16T11:15:37.994 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:37.990+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:38.094 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:38.090+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:38.140 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:38.141 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:38.141 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:38.141 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:38.150 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:38.195 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:38.190+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:38.247 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:38.248 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:38.248 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:38.248 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:38.248 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:38.256 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:38.297 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:38.294+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:38.398 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319087663999994, 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:38.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:38.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.404 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319073205999999, 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.405 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319056005999997, 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:38.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:38.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319042486000001, 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.417 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:38.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:38.422 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319033943999997, 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:38.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:38.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319023799999997, 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:38.430 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:38.394+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:38.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.319010293000005, 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:38.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.440 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318996701000003, 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:38.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318972419000005, 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318938258000003, 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318904860999993, 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:38.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318882216000006, 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:38.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:38.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.466 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318866641, 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.467 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:38.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:38.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.318849607000004, 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:38.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:38.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.315216642999999, 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:38.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:38.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.315182300999993, 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:38.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:38.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.487 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.315166355000002, 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:38.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:38.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.315150743999993, 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:38.493 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:38.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.315135655999995, 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:38.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:38.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.315120854, 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.504 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:38.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.509 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:38.509 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:38.511 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:38.508+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:38.612 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:38.608+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:38.614 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:38.614 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:38.614 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:38.614 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:38.624 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:38.681 INFO:teuthology.orchestra.run.smithi162.stderr:marked out osd.7. 2024-09-16T11:15:38.713 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:38.712+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:38.733 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:38.733 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:38.733 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:38.734 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:38.734 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:38.743 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:38.815 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:38.812+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.703003682, 2024-09-16T11:15:38.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:38.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:38.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:38.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.702985294, 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:38.846 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:38.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.850 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.702684537, 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.70265483199999, 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:38.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.702641434, 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:38.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.863 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.70262762999999, 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:38.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:38.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:38.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.70225263099999, 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:38.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:38.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.6887086, 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.872 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.688687026, 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.876 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:38.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.688670527, 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.881 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:38.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.885 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.688654451, 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:38.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.68827087699999, 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:38.890 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:38.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.894 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.6761887, 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:38.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.676180499, 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.899 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:38.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.903 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.67617247699999, 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:38.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 81.992693598000002, 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.908 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:38.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:38.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.759275247000005, 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.914 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:38.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.919 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 67.372881874000001, 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:38.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:38.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.924 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 65.421277226000001, 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:38.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:38.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 58.285480294000003, 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:38.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:38.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:38.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:38.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:38.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:38.936 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:38.937 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:38.938 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:38.934+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:39.038 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:39.034+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:39.046 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:39.047 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:39.047 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:39.047 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:39.056 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:39.139 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:39.134+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:39.156 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:39.156 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:39.157 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:39.157 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:39.157 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:39.163 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:39.240 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:39.236+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.263 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184953854, 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:39.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184913092000002, 2024-09-16T11:15:39.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184882294000005, 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184863706000002, 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184845654, 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:39.286 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:39.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:39.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:39.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184825727000003, 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:39.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:39.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184806887999997, 2024-09-16T11:15:39.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:39.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.302 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184614938999999, 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:39.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:39.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:39.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.184595658999996, 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.308 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:39.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:39.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:39.313 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.181104090999995, 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:39.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.318 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.181094020000003, 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:39.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:39.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.323 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.181065716999996, 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:39.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:39.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:39.329 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.181051788000005, 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.330 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:39.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:39.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:39.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.336 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.181043024999994, 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:39.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:39.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.181019206000002, 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:39.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.180988084000006, 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:39.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:39.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.180940344999996, 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:39.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:39.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:39.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:39.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:39.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:39.361 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 77.180918237, 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:39.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:39.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.771723872999999, 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.366 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:39.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:39.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 25.771677893, 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:39.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:39.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:39.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:39.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:39.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:39.381 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:39.381 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:39.383 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:39.384 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:39.380+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:39.484 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:39.478+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:39.529 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:39.533 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:39.533 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:39.534 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:39.534 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:39.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:39.547 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:39.567 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:39.585 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:39.580+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:39.585 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:39.608 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:39.630 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:39.639 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:39.640 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:39.649 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:39.653 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:39.679 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:39.682 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:39.683 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:39.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:15:39.698 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:39.696+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:39.751 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:39.751 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:39.760 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:39.786 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:39.782+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:39.857 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:39.858 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:39.867 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:39.887 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:39.884+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:39.966 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:39.966 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:39.975 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:40.018 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:40.016+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:40.089 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:40.084+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:40.119 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.119 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.128 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:40.190 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:40.188+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:40.229 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.229 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.238 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:40.291 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:40.286+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:40.335 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.335 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:40.392 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:40.388+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:40.450 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.450 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.464 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:40.464 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:15:40.526 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:40.522+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:40.592 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.593 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.593 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:40.588+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:40.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:40.694 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:40.692+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:40.717 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.717 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.729 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:40.794 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:40.792+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:40.839 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.839 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.849 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:40.895 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:40.890+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:40.953 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:40.953 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:40.963 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:40.996 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:40.990+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:41.073 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.073 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.083 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:41.125 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:41.122+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:41.198 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:41.194+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:41.235 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.235 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.245 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:41.299 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:41.296+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:41.359 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.360 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.371 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:41.400 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:41.394+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:41.469 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.469 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.484 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:41.500 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:41.494+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:41.581 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.581 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.590 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:41.604 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:41.596+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:41.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:41.702 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:41.700+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:41.745 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.745 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:41.802 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:41.798+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:41.859 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:41.871 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:41.871 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:41.881 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:41.885 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:41.903 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:41.900+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:41.912 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:41.930 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:41.950 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:41.971 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:41.985 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.84801898400001, 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:41.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:41.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:41.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:41.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.84800059600001, 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:41.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:41.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:41.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:41.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.847699839, 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:41.996 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:41.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:41.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:41.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.000 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.847670134, 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.001 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:42.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.847656736, 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:42.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:42.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.010 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.847642932, 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:42.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.015 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.847267933, 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:42.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:42.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.833723902, 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:42.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:42.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:42.025 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.833702328, 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:42.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:42.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:42.030 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.833685829, 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.833669753, 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.036 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:42.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.040 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.833286179, 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.041 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:42.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:42.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:42.045 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.821204002, 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:42.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.049 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.821195801, 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:42.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:42.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:42.054 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.821187779, 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:42.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.137708900000007, 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:42.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:42.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:42.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:42.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.066 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 79.904290548999995, 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:42.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:42.072 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.517897176000005, 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:42.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:42.077 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.566292528000005, 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:42.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 61.430495596, 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:42.083 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:42.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:42.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.088 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.089 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:42.089 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:42.089 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:42.090 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:42.091 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:42.091 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:42.091 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:42.092 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:42.086+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:42.191 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:42.186+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:42.199 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:42.199 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:42.199 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:42.199 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:42.208 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:42.292 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:42.288+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:42.309 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:42.309 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:42.309 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:42.309 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:42.309 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:42.318 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:42.393 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:42.388+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:42.421 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.343105894999994, 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:42.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:42.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.426 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.343065132999996, 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:42.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:42.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.343034334999999, 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.432 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:42.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:42.436 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.343015746999995, 2024-09-16T11:15:42.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:42.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:42.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.342997694999994, 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:42.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:42.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:42.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.342977767999997, 2024-09-16T11:15:42.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:42.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:42.453 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.342958929000005, 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:42.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:42.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.458 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.342766979999993, 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:42.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:42.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.342747700000004, 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:42.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:42.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339256132000003, 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:42.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339246060999997, 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.475 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:42.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339217758000004, 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.480 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:42.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339203828999999, 2024-09-16T11:15:42.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.490 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339195066000002, 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.491 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:42.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:42.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339171246999996, 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.496 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:42.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:42.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.500 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339140125, 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.501 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.503 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:42.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:42.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339092386000004, 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:42.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:42.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:42.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.339070277999994, 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.511 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:42.512 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.513 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.514 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.929875914, 2024-09-16T11:15:42.515 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.516 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:42.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.518 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.519 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:42.520 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.521 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.522 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.929829934000001, 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.523 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.524 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:42.525 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:42.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:42.527 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:42.528 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.529 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.530 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.530 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:42.530 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:15:42.532 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:42.532 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:42.528+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:42.633 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:42.628+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:42.680 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:42.680 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:42.680 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:42.680 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:42.689 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:42.734 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:42.730+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:42.786 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:42.787 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:42.787 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:42.787 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:42.787 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:42.796 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:42.836 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:42.830+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:42.897 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:42.897 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:42.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:42.897 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 142.03116519599999, 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:42.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.86319354699999, 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:42.902 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:42.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:42.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:42.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863181986, 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:42.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:42.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863168784, 2024-09-16T11:15:42.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:42.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863156344, 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:42.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:42.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863142833, 2024-09-16T11:15:42.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:42.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:42.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.86313191799999, 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.925 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.929 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863106, 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:42.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:42.934 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863090546, 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:42.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:42.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.86308065999999, 2024-09-16T11:15:42.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:42.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863070897, 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.944 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:42.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:42.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.949 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.86306071999999, 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:42.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:42.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:42.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863044238, 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:42.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:42.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:42.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:42.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:42.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:42.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:42.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:42.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:42.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:42.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:42.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.961 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863029541, 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:42.962 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:42.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:42.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:42.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:42.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:42.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:42.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.970 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.863011098, 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:42.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:42.975 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.862994149, 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.979 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.862974936, 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.980 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:42.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:42.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:42.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.86295772, 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.985 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:42.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:42.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.86293646599999, 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.990 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:42.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:42.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:42.995 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.862916498, 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.996 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:42.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:42.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:42.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.000 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.002 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:43.003 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:42.998+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:43.102 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:43.098+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:43.105 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:43.105 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:43.105 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:43.105 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.115 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:43.203 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:43.198+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:43.218 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:43.218 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:43.218 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.227 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:43.304 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:43.298+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:43.330 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:43.330 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:43.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 115.128920071, 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:43.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 115.12890400400001, 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:43.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:43.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:43.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:43.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:43.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 115.128887609, 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:43.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:43.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:43.345 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 115.128878035, 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:43.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 115.12886849900001, 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:43.353 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.431031615999999, 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:43.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 71.745196362000001, 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.358 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:43.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:43.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:43.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810677019000003, 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:43.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:43.369 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810596347999997, 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.370 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.375 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810474956, 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810422762000002, 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:43.381 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810366029000001, 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.386 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:43.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.81031883, 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:43.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810268860000001, 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.396 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:43.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.810201745999997, 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.401 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:43.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.808226660000003, 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:43.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.808178175000002, 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:43.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:43.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.808061146999997, 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:43.417 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:43.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:43.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:43.422 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.807969972000002, 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 10.567563125, 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:15:43.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:15:43.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:15:43.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.434 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.436 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:43.436 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:43.430+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:43.506 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:15:43.549 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:43.549 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:43.549 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:43.550 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.559 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:43.561 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:43.558+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:43.638 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:43.636+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:43.656 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:43.656 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:43.656 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:43.656 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:43.657 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.664 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:43.692 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 6, 2, 4] out_osds: [5, 7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:43.692 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:43.693 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 3, 6, 2, 4] 2024-09-16T11:15:43.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-16T11:15:43.739 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:43.734+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:43.813 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:43.813 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:43.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:43.813 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:43.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.813 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 174.032535838, 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:43.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:43.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:43.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:43.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:43.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.821 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64461013799999, 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:43.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:43.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.644458808, 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:43.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.834 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64440732200001, 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:43.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:43.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64433717399999, 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:43.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:43.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64416470399999, 2024-09-16T11:15:43.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:43.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:43.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64410638199999, 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:43.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:43.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:43.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64395426199999, 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:43.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:43.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:43.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:43.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:43.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64377574299999, 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:43.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.874 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.64374276999999, 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.875 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:43.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:43.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.09155504500001, 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.882 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:43.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.887 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.091426248, 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:43.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:43.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.893 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.09136790299999, 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:43.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:43.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.09131351400001, 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:43.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:43.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:43.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.905 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 173.09117920700001, 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.906 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:43.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:43.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 172.309936526, 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:43.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:43.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:43.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:43.917 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 172.30969839799999, 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:43.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:43.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.921125953000001, 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:43.923 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:43.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:194870 4.8 4:13a4eb47:::rbd_data.11b55ebb7981.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 19.360032443000001, 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015060858999999999, 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 194870 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448525+0000", 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.187e-06 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448539+0000", 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3705e-05 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448566+0000", 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6937e-05 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448576+0000", 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0032e-05 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448682+0000", 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106353 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448818+0000", 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135308 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.452297+0000", 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034793049999999998 2024-09-16T11:15:43.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463364+0000", 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011067184000000001 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463431+0000", 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6397999999999997e-05 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463518+0000", 2024-09-16T11:15:43.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7220999999999997e-05 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463584+0000", 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229000000000006e-05 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:195215 4.15 4:ab483f24:::rbd_data.11b55ebb7981.000000000000000a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 18.722238843, 2024-09-16T11:15:43.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015135688, 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 195215 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086318+0000", 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.598e-06 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086326+0000", 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0180000000000001e-06 2024-09-16T11:15:43.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086347+0000", 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0222000000000001e-05 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086359+0000", 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2155e-05 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086451+0000", 2024-09-16T11:15:43.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1815000000000001e-05 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086557+0000", 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010589 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.088907+0000", 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002350651 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:43.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101185+0000", 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01227816 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101257+0000", 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1883000000000003e-05 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101359+0000", 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101766 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:43.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101453+0000", 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3529999999999994e-05 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:43.939 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:43.940 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:43.936+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:43.941 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:44.001 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:15:43.994+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e156 definitely_dead 0 2024-09-16T11:15:44.040 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:44.040 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:44.040 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:44.041 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:44.038+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:44.049 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:44.093 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:44.208 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:44.208 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:44.209 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:44.209 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:44.209 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:44.209 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:44.210 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:44.204+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:44.309 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:44.304+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:44.370 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:44.370 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:44.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:44.370 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:44.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.370 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.19294547699999, 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:44.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.18927396799999, 2024-09-16T11:15:44.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:44.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:44.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.187208676, 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:44.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:44.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.18719482, 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:44.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:44.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:44.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:44.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.18718094499999, 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:44.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.187167178, 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.394 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:44.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.187119669, 2024-09-16T11:15:44.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:44.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.186833851, 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.403 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.094170035, 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.094020853, 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:44.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:44.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:44.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105619633000003, 2024-09-16T11:15:44.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:44.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105572672999998, 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.424 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:44.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:44.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:44.428 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105553981, 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:44.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:44.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105537781999999, 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.433 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:44.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.437 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105522739999998, 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:44.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105506990000002, 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.442 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:44.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:44.446 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105491700000002, 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.450 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105476121999999, 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:44.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:44.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 40.105461142999999, 2024-09-16T11:15:44.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:44.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:44.459 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 11.609971391, 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:44.460 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:44.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:15:44.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:44.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:44.467 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:44.467 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:44.468 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:44.469 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:44.469 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:44.469 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:44.469 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:44.470 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:44.462+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:44.473 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:44.497 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:44.523 INFO:teuthology.orchestra.run.smithi162.stderr:marked out osd.1. 2024-09-16T11:15:44.532 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:44.555 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:44.557 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:44.557 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:44.557 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:44.558 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:44.558 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:44.558 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:44.558 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:44.558 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:44.586 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:44.586 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:44.587 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:44.582+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:44.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:44.669 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:44.662+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:44.739 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:44.739 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:44.748 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:44.770 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:44.768+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:44.847 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:44.847 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:44.856 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:44.870 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:44.866+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:44.955 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:44.955 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:44.964 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:44.971 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:44.968+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:45.062 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.062 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.071 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:45.073 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:45.068+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:45.174 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.174 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.174 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:45.172+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:45.183 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:45.275 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:45.272+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:45.278 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.278 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.288 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:45.375 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:45.372+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:45.382 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.382 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.391 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:45.476 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:45.470+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:45.497 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.498 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.507 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:45.520 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:15:45.577 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:45.570+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:45.615 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.615 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.625 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:45.678 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:45.674+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:45.729 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.729 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.738 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:45.779 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:45.776+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:45.841 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.841 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.850 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:45.881 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:45.873+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:45.952 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:45.952 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:45.961 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:46.006 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:46.004+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:46.059 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.059 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.068 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:46.082 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:46.080+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:46.167 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.168 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:46.183 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:46.177+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:46.273 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.274 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.283 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:46.284 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:46.280+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:46.383 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.383 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.384 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:46.380+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:46.392 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:46.491 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:15:46.492 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:46.481+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:46.506 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.506 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.518 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:46.567 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:46.586 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:46.584+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:46.625 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.625 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.639 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:46.687 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:46.681+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:46.718 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:46.733 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.733 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.742 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:46.742 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:46.762 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:46.788 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:46.789 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:46.781+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:46.812 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:46.838 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:46.849 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:46.849 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:46.859 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:46.860 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:46.889 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:46.890 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:46.888+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:46.892 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:46.893 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:46.977 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:46.977 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:46.977 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:46.978 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:46.989 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:46.990 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:46.988+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:47.092 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:47.088+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:47.106 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:47.106 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:47.106 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:47.106 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:47.106 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:47.117 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:47.192 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:47.188+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.053341938, 2024-09-16T11:15:47.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:47.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:47.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.049670429, 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:47.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:47.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:47.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:47.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.04760513700001, 2024-09-16T11:15:47.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:47.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:47.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.047591281, 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:47.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.251 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.047577406, 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:47.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.047563639, 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:47.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:47.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:47.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.04751613000001, 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:47.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.047230312, 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:47.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:47.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:47.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.954566496, 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:47.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:47.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.954417314, 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.275 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:47.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.966016093999997, 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:47.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:47.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965969133999998, 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:47.285 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:47.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.289 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965950442, 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.290 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:47.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965934243, 2024-09-16T11:15:47.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:47.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:47.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.298 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965919200999998, 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:47.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:47.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:47.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965903451000003, 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.303 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:47.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965888161000002, 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:47.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:47.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965872582999999, 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.312 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:47.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:47.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.965857604, 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:47.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 14.470367852000001, 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:15:47.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:15:47.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.327 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:47.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:47.329 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:47.324+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:47.429 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:47.425+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:47.442 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:47.442 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:47.442 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:47.442 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:47.453 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:47.530 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:47.528+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:47.570 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:47.570 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:47.570 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:47.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:47.632 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:47.628+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:47.690 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:47.690 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.194224709, 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:47.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:47.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:47.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:47.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.940811910999997, 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:47.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:47.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.940542594999997, 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:47.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:47.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:47.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.940144085, 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.707 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:47.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:47.711 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.939855296999994, 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:47.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:47.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.938155881, 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:47.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:47.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.938077565, 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.720 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:47.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.938043825999998, 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:47.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.938003081000005, 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.729 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.934802633000004, 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:47.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:47.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.934784575999998, 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.738 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:47.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:47.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.934758587000005, 2024-09-16T11:15:47.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:47.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:47.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.934060156000001, 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.747 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:47.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:47.751 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.934047597000003, 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:47.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 53.024371844000001, 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:47.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:47.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.341946339000003, 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:47.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:47.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:47.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.772 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.339081446000002, 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:47.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.330035530000004, 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:47.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.785 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.272620025000002, 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:47.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:47.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:47.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 43.22636086, 2024-09-16T11:15:47.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:47.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:47.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:47.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:47.799 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:47.799 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:47.800 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:47.801 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:47.796+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:47.901 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:47.893+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:47.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:47.912 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:47.912 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:47.923 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:48.002 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:48.000+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:48.034 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:48.035 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:48.035 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:48.045 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:48.103 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:48.100+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:48.158 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.077021102000003, 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:48.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:48.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:48.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.077006643999994, 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:48.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076989444000006, 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:48.169 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:48.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076975923999996, 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:48.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:48.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076967382000007, 2024-09-16T11:15:48.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076957238000006, 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.192 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076943731, 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.193 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.197 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076930138999998, 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:48.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:48.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.202 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076905857, 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.207 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076871695999998, 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:48.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.212 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076838299000002, 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.217 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076815654000001, 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.222 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076800078999995, 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.228 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.076783044999999, 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:48.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:48.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.233 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.073150080999994, 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:48.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.238 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.073115739000002, 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.239 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:48.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:48.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.073099792999997, 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:48.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:48.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.073084182000002, 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:48.249 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:48.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.073069094000004, 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:48.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:48.259 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.073054291999995, 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:48.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:48.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:48.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.264 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:48.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:48.266 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:48.264+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:48.366 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:48.364+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4.40f6ded0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.78096429199999995, 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.78098512399999998, 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for peered", 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.386 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 1 2024-09-16T11:15:48.390 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:48.396 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:48.467 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:48.464+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:48.499 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:48.499 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:48.499 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:48.508 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:15:48.509 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:48.568 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:48.564+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:48.633 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:48.633 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:48.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:48.633 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:48.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.49500602099999, 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:48.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.49498763299999, 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:48.639 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.640 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:48.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:48.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:48.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.494686876, 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:48.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:48.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.494657171, 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.651 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:48.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:48.655 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.49464377300001, 2024-09-16T11:15:48.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:48.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.494629969, 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.662 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:48.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.49425497, 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:48.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.48071093900001, 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.673 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:48.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.677 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.480689365, 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.678 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:48.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:48.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:48.683 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.48067286600001, 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:48.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:48.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.48065679, 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.480273216, 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.693 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:48.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50293 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1695744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.468191039, 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050528657999999997, 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50293 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160129+0000", 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:48.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160130+0000", 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3060000000000001e-06 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160133+0000", 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8820000000000001e-06 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160135+0000", 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210365+0000", 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050229993000000001 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210425+0000", 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9750999999999998e-05 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210657+0000", 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023255300000000001 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.702 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.468182838, 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:48.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:48.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 120.468174816, 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:48.707 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:48.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.711 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.784695936999995, 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.712 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:48.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:48.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.551277585999998, 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:48.718 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:48.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:48.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.164884212999993, 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:48.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:48.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:48.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 75.213279564999993, 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:48.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 68.077482633000002, 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:48.736 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:48.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:48.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:48.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:48.742 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:48.744 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:48.744 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:48.737+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:48.844 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:48.837+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:48.855 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:48.855 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:48.855 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:48.865 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:48.908 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:48.946 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:48.944+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:48.964 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:48.965 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:48.965 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:48.965 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:48.965 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:48.973 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:49.044 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:49.046 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:49.044+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:49.069 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:49.078 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000596883, 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.079 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:49.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000556121000002, 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.084 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:49.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:49.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:49.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000525323000005, 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.089 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:49.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000506735000002, 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.094 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:49.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:49.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:49.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000488683, 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:49.099 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:49.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:49.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000468756000004, 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.105 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:49.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000449916999997, 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.111 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:49.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000257968, 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:49.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:49.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 87.000238687999996, 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:49.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:49.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.128 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996747119999995, 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.129 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:49.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:49.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.134 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996737049000004, 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.135 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:49.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996708745999996, 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:49.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.146 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996694817000005, 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:49.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:49.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996686053999994, 2024-09-16T11:15:49.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:49.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:49.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996662235000002, 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.158 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:49.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:49.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.164 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996631113000007, 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:49.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:49.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:49.170 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996583373999997, 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:49.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:49.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:49.176 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 86.996561266000001, 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.177 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:49.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 35.587366901999999, 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:49.181 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:49.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:49.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:49.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:49.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:49.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:49.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 35.587320922000004, 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:49.190 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:49.191 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:49.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:49.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:49.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:49.198 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:49.200 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:49.200 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:49.201 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:49.201 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:49.201 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:49.201 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:49.202 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:49.196+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:49.202 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:49.204 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:49.204 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:49.204 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:49.204 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:49.204 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:49.204 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:49.205 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:49.205 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:49.301 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:49.296+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:49.307 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.307 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:49.402 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:49.397+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:49.417 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.417 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.426 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:49.502 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:49.497+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:49.528 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.528 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.534 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4] out_osds: [5, 7, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:49.534 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:49.534 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T11:15:49.534 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-16T11:15:49.535 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T11:15:49.538 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:49.553 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:15:49.603 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:49.600+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:49.632 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.632 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.640 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:49.704 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:49.697+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:49.736 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.736 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.745 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:49.805 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:49.800+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:49.846 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.846 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.855 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:49.906 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:49.904+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:49.957 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:49.957 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:49.966 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:50.006 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:50.001+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:50.064 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.064 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.073 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:50.107 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:50.101+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:50.174 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.174 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.183 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:50.230 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:50.225+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:50.292 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.293 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:50.309 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:50.304+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:50.397 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.397 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.410 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:50.405+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:50.413 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:50.457 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T11:15:50.513 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:50.509+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:50.528 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.528 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.538 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:50.611 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:50.605+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:50.632 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.632 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:50.713 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:50.708+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:50.742 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.742 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.751 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:50.814 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:50.812+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:50.847 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.847 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.857 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:50.915 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:50.912+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:50.954 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:50.954 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:50.963 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:51.016 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:51.009+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:51.058 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:51.058 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:51.067 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:51.117 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:51.113+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:51.164 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:51.164 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:51.173 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:51.215 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:51.217 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:51.213+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:51.275 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:51.276 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:51.285 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:51.319 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:51.313+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:51.352 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:51.371 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:51.379 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:51.379 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:51.388 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:51.391 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:51.409 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:51.420 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:51.416+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:51.432 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:51.458 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:51.485 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:51.508 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.37054814299999, 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:51.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.37052975500001, 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.513 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:51.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.517 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.370228998, 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:51.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.521 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.370199293, 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.522 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:51.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.37018589500001, 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.526 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:51.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.530 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.370172091, 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:51.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.534 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.369797092, 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:51.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:51.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.356253061, 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:51.539 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:51.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:51.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.356231487, 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:51.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:51.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.356214988, 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.548 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:51.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:51.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.356198912, 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:51.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:51.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:51.558 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.355815338, 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.559 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:51.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.563 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.34372496, 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.564 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:51.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:51.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.343716938, 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.569 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:51.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:51.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.573 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 94.660238058999994, 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.574 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:51.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.426819707999996, 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:51.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:51.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:51.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:51.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.040426335000006, 2024-09-16T11:15:51.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:51.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:51.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:51.591 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 78.088821687000006, 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.592 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:51.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:51.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:51.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:51.599 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.953024755000001, 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:51.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 3.9003008370000001, 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:51.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:15:51.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.612 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.613 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:51.613 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:51.613 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:51.613 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:15:51.616 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:51.616 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:51.612+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:51.616 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:51.616 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:51.616 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:51.616 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:51.616 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:51.617 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:51.617 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:51.617 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:51.618 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:51.716 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:51.709+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:51.729 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:51.729 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:51.729 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:51.739 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:51.816 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:51.809+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:51.839 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:51.839 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:51.839 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:51.839 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:51.840 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:51.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:51.917 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:51.909+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874534842000003, 2024-09-16T11:15:51.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:15:51.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:15:51.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.956 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874494080000005, 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.957 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:15:51.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:15:51.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874463281999994, 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.962 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874444694000005, 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.967 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:15:51.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:15:51.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:51.972 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874426642000003, 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:15:51.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874406715000006, 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:15:51.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.982 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874387876, 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.983 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:15:51.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874195927000002, 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.988 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:15:51.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:15:51.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.993 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.874176646999999, 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:51.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:51.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:15:51.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:51.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870685078999998, 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:51.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:15:52.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.003 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870675008000006, 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:15:52.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:15:52.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:15:52.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870646704999999, 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:52.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:15:52.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.014 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870632775999994, 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:52.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.019 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870624012999997, 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:15:52.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:15:52.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870600194000005, 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.025 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:52.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870569071999995, 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.030 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.031 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:15:52.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:15:52.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870521332999999, 2024-09-16T11:15:52.035 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:15:52.037 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.038 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:15:52.039 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.040 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.870499225000003, 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.041 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:15:52.042 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.043 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.044 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.461304861000002, 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.046 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.047 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:15:52.048 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.049 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:15:52.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.461258880999999, 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.053 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:15:52.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.061 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.061 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.062 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:52.062 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:52.057+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:52.163 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:52.157+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:52.167 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:52.167 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:52.167 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:52.167 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.176 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:52.264 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:52.257+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:52.277 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:52.277 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:52.277 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:52.277 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:52.277 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.289 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:52.365 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:52.360+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:52.393 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:52.393 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:52.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:52.393 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:52.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.52830923900001, 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:15:52.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:15:52.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:52.398 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36033759, 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:52.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:15:52.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36032602900001, 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.403 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:15:52.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:15:52.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360312827, 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:15:52.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:15:52.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.415 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360300387, 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:15:52.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360286876, 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:15:52.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:52.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:15:52.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360275961, 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.426 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:15:52.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:15:52.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:15:52.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36025004299999, 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:15:52.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:15:52.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360234589, 2024-09-16T11:15:52.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:15:52.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360224703, 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:15:52.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36021494000001, 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.446 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:15:52.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:15:52.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360204763, 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:15:52.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:15:52.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:15:52.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36018828100001, 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:15:52.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:15:52.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36017358399999, 2024-09-16T11:15:52.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:15:52.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:15:52.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36015514100001, 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:52.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:15:52.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36013819199999, 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.468 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:15:52.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:15:52.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:52.472 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.36011897900001, 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:15:52.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:15:52.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360101763, 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.477 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:15:52.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:15:52.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360080509, 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:15:52.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:15:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.360060541, 2024-09-16T11:15:52.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:15:52.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:15:52.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:15:52.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:15:52.493 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.494 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.494 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.494 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.496 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:15:52.496 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:52.496 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:52.492+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:52.596 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:52.592+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:52.645 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:52.645 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:52.645 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:52.645 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.654 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:52.697 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:52.689+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:52.750 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:52.750 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:52.750 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:52.750 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:52.750 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.759 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:52.798 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:52.796+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:52.859 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.659544764, 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:52.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.864 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.659528697, 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.865 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:15:52.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:15:52.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.659512302, 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:52.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:15:52.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.65950272800001, 2024-09-16T11:15:52.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:15:52.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:52.877 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 124.659493192, 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:15:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:52.881 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 95.961656309000006, 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:15:52.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.885 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 81.275821054999994, 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:52.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.891 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.341301712000003, 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.892 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:15:52.896 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.341221040999997, 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.897 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.905 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.341099649, 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:52.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:15:52.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:52.910 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.341047455000002, 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:15:52.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:15:52.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:15:52.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.340990722000001, 2024-09-16T11:15:52.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:15:52.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:15:52.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.340943523, 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:15:52.920 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:15:52.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.340893553000001, 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:15:52.925 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:15:52.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.340826438999997, 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.930 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.338851353000003, 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.935 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.338802868000002, 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.940 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:15:52.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:15:52.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:15:52.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.338685839999997, 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:15:52.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.338594665000002, 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:15:52.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:15:52.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 20.098187818, 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:52.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:15:52.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:15:52.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.961 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:52.962 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:52.962 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:52.963 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:52.963 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:52.960+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:53.064 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:53.057+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:53.115 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:53.115 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:53.115 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:15:53.115 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:53.124 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:53.165 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:53.161+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:53.227 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:53.227 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:15:53.227 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:15:53.227 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:15:53.228 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:53.236 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:53.266 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:53.261+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.55535629299999, 2024-09-16T11:15:53.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:53.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:53.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16743059300001, 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:53.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16727926300001, 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:53.349 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:53.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.355 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16722777699999, 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:53.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:53.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.361 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.167157629, 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:53.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:53.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:53.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16698515900001, 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:53.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:53.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:53.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16692683700001, 2024-09-16T11:15:53.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:53.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:53.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:53.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16677471700001, 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:53.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.16659619800001, 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.385 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:53.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:53.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 183.166563225, 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:53.391 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:53.397 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 182.61437549999999, 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:53.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:53.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 182.61424670299999, 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:53.403 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:53.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:53.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 182.61418835800001, 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.409 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:53.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:53.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 182.61413396899999, 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.415 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:53.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:53.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:53.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 182.613999662, 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.426 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 181.83275698099999, 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:53.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:53.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.432 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 181.83251885300001, 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.433 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:53.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.438 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 96.443946408000002, 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.442 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:194870 4.8 4:13a4eb47:::rbd_data.11b55ebb7981.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.882852897999999, 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015060858999999999, 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 194870 2024-09-16T11:15:53.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448525+0000", 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.187e-06 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448539+0000", 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3705e-05 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448566+0000", 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6937e-05 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448576+0000", 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0032e-05 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448682+0000", 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106353 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448818+0000", 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135308 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.452297+0000", 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034793049999999998 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463364+0000", 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011067184000000001 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463431+0000", 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6397999999999997e-05 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463518+0000", 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7220999999999997e-05 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463584+0000", 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229000000000006e-05 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.448 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:195215 4.15 4:ab483f24:::rbd_data.11b55ebb7981.000000000000000a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 28.245059298000001, 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015135688, 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 195215 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086318+0000", 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.598e-06 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086326+0000", 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0180000000000001e-06 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086347+0000", 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0222000000000001e-05 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086359+0000", 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2155e-05 2024-09-16T11:15:53.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086451+0000", 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1815000000000001e-05 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086557+0000", 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010589 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:53.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.088907+0000", 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002350651 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101185+0000", 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01227816 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101257+0000", 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1883000000000003e-05 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101359+0000", 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101766 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101453+0000", 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3529999999999994e-05 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:53.454 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:53.456 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:53.452+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:53.456 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:53.557 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:53.552+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:53.576 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198880 4.16 4:6efdc951:::rbd_header.11b5f06e589e:head [watch unwatch cookie 140364698958864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e166)", 2024-09-16T11:15:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:53.573288+0000", 2024-09-16T11:15:53.576 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.001401394, 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001420569, 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for sub ops", 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198880 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:53.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573288+0000", 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573288+0000", 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:53.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573289+0000", 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4799999999999998e-07 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573291+0000", 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.6120000000000001e-06 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573302+0000", 2024-09-16T11:15:53.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.026e-05 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573305+0000", 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5590000000000001e-06 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573524+0000", 2024-09-16T11:15:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00021921000000000001 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573587+0000", 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2830000000000007e-05 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573678+0000", 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0821000000000003e-05 2024-09-16T11:15:53.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574367+0000", 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00068912400000000005 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198881 4.16 4:6efdc951:::rbd_header.11b5f06e589e:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:53.573697+0000", 2024-09-16T11:15:53.582 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 0.00099252600000000009, 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010758829999999999, 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "waiting for rw locks", 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198881 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573697+0000", 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573697+0000", 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:53.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573698+0000", 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1370000000000001e-06 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573704+0000", 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5099999999999998e-06 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573713+0000", 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2480000000000002e-06 2024-09-16T11:15:53.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.573718+0000", 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4739999999999996e-06 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:53.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574097+0000", 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037871600000000002 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for rw locks", 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574159+0000", 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2198000000000003e-05 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_repop_reply(client.4533.0:198880 4.16 e166/31 ondisk, result = 0)", 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:53.574685+0000", 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 4.194e-06, 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000125666, 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:53.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574685+0000", 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574685+0000", 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574686+0000", 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1900000000000002e-07 2024-09-16T11:15:53.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574690+0000", 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2590000000000002e-06 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574704+0000", 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3305999999999999e-05 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574709+0000", 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8540000000000002e-06 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574757+0000", 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7595000000000001e-05 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:53.574765+0000", 2024-09-16T11:15:53.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1769999999999998e-06 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 3 2024-09-16T11:15:53.592 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:53.593 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:53.620 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:53.666 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:53.661+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:53.700 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:53.700 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:53.701 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:53.701 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:53.701 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:53.711 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:53.759 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:53.759 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:53.753+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:53.781 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:53.799 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:53.807 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:53.807 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:53.817 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:53.818 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:53.837 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:53.860 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:53.861 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:53.856+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:53.887 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:53.909 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:53.909 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:53.912 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:53.919 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:53.962 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:53.960+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:54.014 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.014 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.023 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:54.063 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:54.057+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:54.125 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.126 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.134 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:54.164 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:54.160+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:54.235 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.236 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.244 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:54.265 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:54.260+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:54.344 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.344 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.353 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:54.397 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:54.393+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:54.452 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.452 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.462 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:54.466 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:54.461+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:54.498 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:15:54.570 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:54.564+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:54.594 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.595 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.605 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:54.669 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:54.664+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:54.715 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.715 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.724 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:54.769 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:54.764+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:54.819 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.819 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.829 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:54.870 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:54.865+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:54.937 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:54.937 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:54.947 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:54.971 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:54.968+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:55.052 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.052 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.061 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:55.071 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:55.068+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:55.158 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.159 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.167 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:55.172 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:55.165+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:55.273 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:55.269+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:55.319 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.319 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.328 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:55.374 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:55.369+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:55.422 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.422 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.432 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:55.465 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:15:55.472 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4] out_osds: [5, 7, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:15:55.472 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:15:55.472 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:15:55.472 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-16T11:15:55.475 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:55.469+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:55.535 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.535 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:55.576 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:55.572+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:55.643 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.643 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.653 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:55.698 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:55.696+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:55.757 INFO:tasks.ceph.mgr.y.smithi162.stderr:2024-09-16T11:15:55.749+0000 7f228efd7700 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-16T11:15:55.758 INFO:teuthology.orchestra.run.smithi162.stdout:moved 2 / 90 (2.22222%) 2024-09-16T11:15:55.758 INFO:teuthology.orchestra.run.smithi162.stdout:avg 18 2024-09-16T11:15:55.758 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.60768 -> 2.28035 (expected baseline 3.79473) 2024-09-16T11:15:55.758 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 14 -> 14 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:55.758 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 22 -> 20 pgs (1.22222 -> 1.11111 * mean) 2024-09-16T11:15:55.758 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:55.759 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:55.759 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:55.759 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:55.759 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0050 2024-09-16T11:15:55.760 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0055 2024-09-16T11:15:55.760 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 weight 1.0000 -> 0.8876 2024-09-16T11:15:55.761 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:55.771 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:55.771 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T11:15:55.778 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:55.773+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:55.798 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.798 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.808 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:55.879 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:55.876+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:55.906 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:55.906 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:55.915 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:55.928 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:15:55.980 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:55.976+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:56.011 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:56.011 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:56.020 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:56.058 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:56.080 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:56.080 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:56.073+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:56.105 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:56.124 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:56.146 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:56.165 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:56.165 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:56.170 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:56.175 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:56.180 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:56.176+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:56.193 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:56.214 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:56.216 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:56.216 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:56.216 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:56.216 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:56.217 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:56.217 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:56.217 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:56.217 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.49416090400001, 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:15:56.275 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:56.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:15:56.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:15:56.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.106235204, 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:15:56.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.10608387400001, 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:15:56.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:15:56.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:15:56.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:56.297 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.10603238799999, 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:56.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:15:56.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:15:56.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:15:56.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.303 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.10596224, 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.304 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.305 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:15:56.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.305 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:15:56.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.10578977, 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:15:56.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.317 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.105731448, 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:15:56.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:56.323 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.105579328, 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:56.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:15:56.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:15:56.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.105400809, 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.330 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:15:56.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.335 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 186.105367836, 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.336 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:15:56.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:15:56.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 185.55318011099999, 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:15:56.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:15:56.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 185.55305131399999, 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:15:56.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:15:56.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 185.552992969, 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.354 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:15:56.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:15:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 185.55293857999999, 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.360 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:15:56.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 185.55280427299999, 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:15:56.366 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:15:56.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:15:56.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:15:56.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:15:56.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:15:56.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:56.375 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 184.77156159200001, 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:15:56.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:15:56.381 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 184.77132346400001, 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:56.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:15:56.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:15:56.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.382751018999997, 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:56.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:15:56.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:194870 4.8 4:13a4eb47:::rbd_data.11b55ebb7981.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T11:15:56.392 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 31.821657509000001, 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015060858999999999, 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 194870 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:56.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448525+0000", 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.187e-06 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448539+0000", 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3705e-05 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448566+0000", 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6937e-05 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448576+0000", 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0032e-05 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448682+0000", 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106353 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448818+0000", 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135308 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.452297+0000", 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034793049999999998 2024-09-16T11:15:56.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463364+0000", 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011067184000000001 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463431+0000", 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6397999999999997e-05 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463518+0000", 2024-09-16T11:15:56.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7220999999999997e-05 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463584+0000", 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229000000000006e-05 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:195215 4.15 4:ab483f24:::rbd_data.11b55ebb7981.000000000000000a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:56.398 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 31.183863908999999, 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015135688, 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 195215 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086318+0000", 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.598e-06 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:15:56.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086326+0000", 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0180000000000001e-06 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086347+0000", 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0222000000000001e-05 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086359+0000", 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2155e-05 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086451+0000", 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1815000000000001e-05 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086557+0000", 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010589 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.088907+0000", 2024-09-16T11:15:56.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002350651 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101185+0000", 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01227816 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101257+0000", 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1883000000000003e-05 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:15:56.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101359+0000", 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101766 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101453+0000", 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3529999999999994e-05 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:15:56.404 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:15:56.406 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:56.404+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:56.406 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:56.507 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:56.504+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:56.515 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:56.515 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:56.516 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:56.516 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:56.525 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:56.570 INFO:teuthology.orchestra.run.smithi162.stdout:moved 2 / 90 (2.22222%) 2024-09-16T11:15:56.570 INFO:teuthology.orchestra.run.smithi162.stdout:avg 18 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.60768 -> 2.28035 (expected baseline 3.79473) 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 14 -> 14 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 22 -> 20 pgs (1.22222 -> 1.11111 * mean) 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0050 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0055 2024-09-16T11:15:56.571 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 weight 1.0000 -> 0.8876 2024-09-16T11:15:56.592 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:56.592 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-16T11:15:56.608 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:56.601+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:56.643 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:56.643 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:56.644 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:56.644 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:56.644 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:56.654 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:56.709 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:56.704+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:56.791 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.612700773, 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.792 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:15:56.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.796 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.60902926399999, 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:15:56.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:15:56.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.60696397199999, 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.801 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:15:56.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.805 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.60695011600001, 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:15:56.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.606936241, 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.810 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:56.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:15:56.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.60692247399999, 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:15:56.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.606874965, 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.819 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:15:56.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.60658914699999, 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:15:56.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:15:56.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.513925331, 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:15:56.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:15:56.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:15:56.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 132.51377614899999, 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:15:56.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525374929000002, 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:15:56.840 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:15:56.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525327969000003, 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:15:56.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525309276999998, 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.849 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:15:56.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:56.853 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525293077999997, 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:15:56.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:15:56.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525278036000003, 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:15:56.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525262286, 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:15:56.862 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:15:56.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:15:56.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525246996, 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.867 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:15:56.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525231417999997, 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:56.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:15:56.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:15:56.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.525216438999998, 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:15:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 24.029726687, 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:15:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:15:56.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:15:56.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:15:56.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:56.887 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:56.887 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:56.888 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:avg 18 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.28035 -> 2.28035 (expected baseline 3.79473) 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 14 -> 14 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 20 -> 20 pgs (1.11111 -> 1.11111 * mean) 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 3 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0050 2024-09-16T11:15:56.889 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0503 2024-09-16T11:15:56.890 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:56.890 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T11:15:56.891 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-16T11:15:56.891 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:56.881+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:56.989 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:56.981+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:57.035 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:57.036 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:57.036 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:57.044 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:15:57.089 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:57.085+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:57.132 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:57.132 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:57.132 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:57.141 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:15:57.191 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:57.185+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:moved 0 / 90 (0%) 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:avg 18 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.28035 -> 2.28035 (expected baseline 3.79473) 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 14 -> 14 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 20 -> 20 pgs (1.11111 -> 1.11111 * mean) 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:oload 1000 2024-09-16T11:15:57.232 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:57.233 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:57.233 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0050 2024-09-16T11:15:57.233 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0503 2024-09-16T11:15:57.236 INFO:teuthology.orchestra.run.smithi162.stderr:no change 2024-09-16T11:15:57.239 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:57.239 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 109.742086442, 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.240 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:15:57.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:15:57.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:15:57.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.247 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.488673644000002, 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:15:57.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.252 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.488404328000001, 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:15:57.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.488005818000005, 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.257 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:15:57.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:15:57.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.487717029999999, 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.262 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:15:57.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.486017614000005, 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:15:57.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:15:57.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.485939298000005, 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:15:57.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.485905559000003, 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:15:57.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:15:57.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:15:57.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.485864813999996, 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:15:57.281 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:15:57.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.285 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.482664365999995, 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:15:57.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:15:57.290 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.482646309000003, 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.291 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.295 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.482620319999995, 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.296 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:15:57.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:15:57.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:57.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.481921889000006, 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:15:57.304 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.481909329999993, 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.305 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:15:57.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.309 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.572233576999999, 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.310 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:15:57.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:15:57.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.889808072000001, 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.318 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:15:57.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:15:57.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.886943178999999, 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:57.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.877897263000001, 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:15:57.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:15:57.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:15:57.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.820481758, 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:15:57.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:15:57.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:15:57.345 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 52.774222592999998, 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:57.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:15:57.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:15:57.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.352 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:57.353 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T11:15:57.354 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T11:15:57.355 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:15:57.355 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:57.349+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:57.454 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:57.452+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:57.482 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:15:57.510 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.510 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:57.510 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:57.510 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:57.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:15:57.569 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:57.565+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:57.656 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:57.652+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:57.663 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:57.663 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:57.663 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:57.672 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:15:57.756 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:57.752+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:57.773 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:57.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691519604000007, 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.775 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:15:57.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691505145999997, 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.779 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691487945999995, 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:15:57.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:15:57.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.789 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691474425999999, 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:15:57.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.794 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691465883999996, 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:15:57.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.798 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:15:57.799 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691455739999995, 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.800 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:15:57.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:15:57.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.805 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691442233000004, 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:15:57.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:15:57.810 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691428641000002, 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.811 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:15:57.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691404359000003, 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.817 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:15:57.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:57.822 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691370198000001, 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:57.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:15:57.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:15:57.827 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691336801000006, 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.828 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:15:57.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:15:57.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:15:57.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691314156000004, 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.834 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:15:57.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691298580999998, 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:15:57.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:15:57.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.691281547000003, 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.845 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:15:57.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:15:57.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.687648582999998, 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.850 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:15:57.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:15:57.855 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.687614241000006, 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.856 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.687598295000001, 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:15:57.861 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:15:57.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:15:57.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.687582684000006, 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.867 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:15:57.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.687567595999994, 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:15:57.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:15:57.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:15:57.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.687552793999998, 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:15:57.878 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:15:57.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:57.883 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:57.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:15:57.886 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:57.880+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:57.986 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:57.981+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:57.994 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:57.994 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:57.994 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:15:57.994 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:58.004 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:15:58.086 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:58.084+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:58.146 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:15:58.146 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:15:58.147 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:15:58.147 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:58.155 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:15:58.187 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:58.181+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:58.233 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:15:58.253 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:15:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:15:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:15:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:15:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.253 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.11430512699999, 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.257 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.11428673899999, 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:15:58.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:15:58.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.113985982, 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:15:58.262 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:15:58.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.113956277, 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:15:58.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:15:58.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:15:58.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.11394287900001, 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.271 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:58.275 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.11392907499999, 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:15:58.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.279 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.11355407600001, 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:15:58.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.100010045, 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.284 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:58.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.09998847099999, 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:58.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:15:58.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:15:58.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.09997197199999, 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:15:58.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.09995589600001, 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:15:58.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:15:58.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:58.301 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.099572322, 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:15:58.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.305 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.08748194399999, 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:15:58.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:15:58.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 130.08747392199999, 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:15:58.310 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:15:58.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 101.40399504299999, 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:15:58.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:15:58.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:15:58.320 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 96.170576691999997, 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:58.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.784183318999993, 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.326 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.328 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:15:58.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:58.331 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.832578670999993, 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:15:58.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 77.696781739000002, 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.337 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:15:58.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:15:58.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 10.644057821000001, 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:15:58.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:15:58.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:15:58.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:15:58.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:15:58.349 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:15:58.351 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:15:58.351 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:58.345+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:58.367 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:15:58.388 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:15:58.410 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:15:58.431 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:15:58.451 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:58.445+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:58.452 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:15:58.469 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:58.469 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:58.477 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:15:58.478 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:15:58.498 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:15:58.519 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:15:58.521 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:15:58.522 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:15:58.577 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:58.577 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:58.578 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:58.573+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:moved 2 / 90 (2.22222%) 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:avg 18 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:stddev 2.28035 -> 2.44949 (expected baseline 3.79473) 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:min osd.2 with 14 -> 14 pgs (0.777778 -> 0.777778 * mean) 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:max osd.3 with 20 -> 18 pgs (1.11111 -> 1 * mean) 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:15:58.585 INFO:teuthology.orchestra.run.smithi162.stdout:oload 110 2024-09-16T11:15:58.586 INFO:teuthology.orchestra.run.smithi162.stdout:max_change 1 2024-09-16T11:15:58.586 INFO:teuthology.orchestra.run.smithi162.stdout:max_change_osds 4 2024-09-16T11:15:58.586 INFO:teuthology.orchestra.run.smithi162.stdout:average_utilization 0.0043 2024-09-16T11:15:58.586 INFO:teuthology.orchestra.run.smithi162.stdout:overload_utilization 0.0047 2024-09-16T11:15:58.586 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3 weight 0.8876 -> 0.7782 2024-09-16T11:15:58.592 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:15:58.603 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:15:58.653 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:15:58.648+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:15:58.711 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:58.711 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:58.721 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:15:58.754 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:58.749+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:58.823 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:58.823 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:58.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:15:58.855 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:58.849+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:58.936 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:58.936 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:58.946 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:15:58.956 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:58.952+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:59.042 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.043 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.051 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:15:59.057 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:15:59.052+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:15:59.144 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.144 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:15:59.158 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:59.156+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:15:59.249 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.249 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.258 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:15:59.260 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:59.253+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:59.357 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.358 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.360 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:15:59.353+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:15:59.366 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:15:59.461 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:15:59.457+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:15:59.467 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.467 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.476 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:15:59.563 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:15:59.560+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:15:59.577 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.578 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.588 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:15:59.664 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:15:59.657+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:15:59.685 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.685 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.695 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:15:59.765 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:59.757+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:59.802 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.803 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.812 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:15:59.866 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:15:59.861+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:15:59.910 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:15:59.910 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:15:59.919 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:15:59.966 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:15:59.964+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:00.067 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:00.061+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:00.073 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:00.073 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:00.082 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:00.167 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:00.164+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:00.178 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:00.178 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:00.187 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:00.268 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:00.261+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:00.281 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:00.281 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:00.291 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:00.369 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:00.364+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:00.431 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:00.432 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:00.440 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:00.470 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:00.465+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:00.533 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:00.571 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:00.568+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:00.572 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:00.572 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:00.585 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:00.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:16:00.672 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:00.665+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:00.693 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:00.723 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:00.739 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:00.759 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:00.759 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:00.759 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:00.759 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:00.759 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:00.759 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:00.769 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:00.773 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:00.768+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:00.787 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:00.820 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:00.852 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:00.865 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784430315999998, 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.866 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:00.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784415858000003, 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.871 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:00.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:00.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:00.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784398658000001, 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.876 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:00.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784385138000005, 2024-09-16T11:16:00.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:00.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:00.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784376596000001, 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:00.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:00.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.891 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784366452, 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:00.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:00.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784352944999995, 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:00.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:00.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.900 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.901 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784339353000007, 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.902 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:00.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.906 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784315070999995, 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:00.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.909 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.911 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784280910000007, 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.912 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:00.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784247512999997, 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:00.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.919 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:00.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784224867999995, 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:00.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:00.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:00.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.927 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784209293000004, 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.928 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:00.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.784192258999994, 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:00.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.937 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.780559295000003, 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.938 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:00.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.942 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.780524952999997, 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.943 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:00.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.947 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.780509007000006, 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:00.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.950 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:00.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.780493395999997, 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.953 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:00.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:00.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.780478307999999, 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:00.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:00.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:00.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 98.780463506000004, 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:00.963 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:00.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:00.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:00.968 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:00.970 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:00.970 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:00.971 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:16:00.972 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:00.968+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:01.071 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:01.068+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:01.115 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:01.115 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:01.115 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:01.115 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:01.124 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:01.172 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:01.168+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:01.228 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:01.228 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:01.228 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:01.228 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:01.228 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:01.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:01.285 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:01.281+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:01.328 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:01.328 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:01.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.19051075199999, 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.329 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:01.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.19049236399999, 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.190191607, 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:01.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:01.342 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.190161902, 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.346 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.19014850400001, 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.347 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:01.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.19013469999999, 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.352 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:01.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:01.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.18975970100001, 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.357 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:01.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.17621567, 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.362 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:01.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:01.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.17619409599999, 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:01.366 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:01.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.17617759699999, 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:01.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.17616152100001, 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.375 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:01.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:01.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.175777947, 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.380 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.16368756899999, 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:01.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.388 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.16367954699999, 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.390 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:01.392 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.48020066799999, 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.393 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:01.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:01.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:01.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.246782316999997, 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:01.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.860388943999993, 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.404 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:01.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 87.908784295999993, 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.409 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:01.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:01.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.772987364000002, 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.415 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:01.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:01.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 13.720263446000001, 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:01.420 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:01.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:01.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:01.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:01.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:01.426 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:01.427 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:01.428 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:01.428 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:01.421+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:01.529 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:01.524+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:01.537 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:01.537 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:01.537 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:01.537 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:01.547 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:01.613 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:16:01.630 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:01.628+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:01.648 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:01.648 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:01.648 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:01.648 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:01.648 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:01.657 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:01.731 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:01.725+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684365670000005, 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.761 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:01.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:01.766 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684324907999994, 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:01.771 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684294109999996, 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:01.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:01.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:01.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684275521999993, 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.777 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:01.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:01.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684257470000006, 2024-09-16T11:16:01.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:01.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684237542999995, 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.788 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:01.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684218704000003, 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.794 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:01.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:01.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:01.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684026755000005, 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.799 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.684007475000001, 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:01.804 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:01.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680515907, 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.809 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680505835999995, 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:01.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680477533000001, 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.820 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:01.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.824 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680463603999996, 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.825 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:01.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680454841, 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:01.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680431021999993, 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.835 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:01.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680399899999998, 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.840 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:01.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680352161000002, 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:01.845 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:01.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:01.850 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 99.680330053000006, 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:01.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:01.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:01.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:01.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.271135688999998, 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.856 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:01.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:01.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 48.271089709000002, 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:01.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:01.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:01.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:01.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:01.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:01.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:01.871 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:01.872 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:01.873 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:01.865+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:01.973 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:01.969+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:01.980 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:01.980 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:01.980 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:01.980 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:01.989 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:02.074 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:02.069+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:02.091 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.091 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:02.091 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:02.091 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:02.091 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.100 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:02.176 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:16:02.172+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 161.337472015, 2024-09-16T11:16:02.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:16:02.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:16:02.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:16:02.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.210 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16950036599999, 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:02.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:16:02.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:16:02.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:16:02.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16948880499999, 2024-09-16T11:16:02.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:16:02.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.169475603, 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:16:02.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:16:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16946316299999, 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.226 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:16:02.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:02.230 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.169449652, 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:16:02.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:16:02.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16943873700001, 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:02.239 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.169412819, 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:02.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.243 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16939736500001, 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:16:02.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:16:02.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16938747899999, 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.248 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:02.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.252 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16937771600001, 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:16:02.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16936753900001, 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.257 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.261 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.169351057, 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:02.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16933635999999, 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:16:02.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.270 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.169317917, 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:02.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16930096799999, 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:16:02.275 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:16:02.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.279 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16928175499999, 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.280 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16926453900001, 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.284 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:16:02.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16924328499999, 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.289 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:16:02.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.293 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.16922331699999, 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:02.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.298 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.298 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.299 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:02.300 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:02.293+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:02.400 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:02.397+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:02.409 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.409 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:02.409 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:02.409 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.418 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:02.502 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:02.497+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:02.520 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.521 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:02.521 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:02.521 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:02.521 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.530 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:02.602 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:02.597+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:02.642 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.642 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:02.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:02.642 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:02.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.642 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.44191023299999, 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.643 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:02.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:02.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.441894166, 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.647 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:02.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.44187777100001, 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.651 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.44186819699999, 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.656 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:02.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.657 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:02.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.657 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.441858661, 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.661 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 105.744021778, 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.665 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:02.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:02.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:02.669 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 91.058186524000007, 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:02.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:02.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123667181000002, 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.676 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:02.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:02.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123586510000003, 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.681 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:02.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.685 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123465117999999, 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.686 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:02.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123412924, 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.691 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:02.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123356190999999, 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:02.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123308991999998, 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:02.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:02.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123259021999999, 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:02.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.123191908000003, 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:02.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:02.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.121216822000001, 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.718 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.723 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.121168337, 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.726 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.728 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.121051309000002, 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:02.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.120960134000001, 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:02.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:02.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 29.880553287000001, 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:02.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:02.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:02.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:02.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:02.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:02.745 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.747 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:16:02.747 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:02.744+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:02.839 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.840 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:02.840 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:02.840 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.847 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:02.841+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:02.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:16:02.943 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:02.943 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:02.943 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:02.943 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:02.943 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:02.947 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:16:02.944+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:16:02.951 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:16:02.972 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:03.048 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:03.041+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:03.050 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:03.050 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:03.050 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:03.050 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:03.050 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 193.26998936699999, 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:16:03.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:16:03.053 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.054 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.056 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:16:03.057 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88206366700001, 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.058 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:16:03.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.060 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.061 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.062 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.063 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88191233699999, 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.064 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.065 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.066 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:16:03.067 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:16:03.068 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.881860851, 2024-09-16T11:16:03.069 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.070 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.071 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.072 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:16:03.073 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.074 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88179070300001, 2024-09-16T11:16:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.076 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:16:03.077 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.078 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:16:03.079 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:16:03.080 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88161823300001, 2024-09-16T11:16:03.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:16:03.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.087 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88155991100001, 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.088 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:16:03.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:16:03.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.095 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88140779099999, 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:16:03.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.100 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.101 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88122927200001, 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:16:03.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:16:03.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.88119629900001, 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.107 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:16:03.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.113 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.329008574, 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:16:03.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.119 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.328879777, 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.120 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.121 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:16:03.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:16:03.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:16:03.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.126 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:03.127 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.32882143200001, 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:03.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:16:03.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.328767043, 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:16:03.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:03.139 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 192.328632736, 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:16:03.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:16:03.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:16:03.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 191.54739005499999, 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.148 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:16:03.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:16:03.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 191.54715192699999, 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:03.152 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 106.15857948199999, 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:194870 4.8 4:13a4eb47:::rbd_data.11b55ebb7981.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 38.597485972000001, 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015060858999999999, 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.162 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 194870 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:16:03.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448525+0000", 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.187e-06 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448539+0000", 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3705e-05 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448566+0000", 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6937e-05 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448576+0000", 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0032e-05 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448682+0000", 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106353 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448818+0000", 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135308 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.452297+0000", 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034793049999999998 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463364+0000", 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011067184000000001 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463431+0000", 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6397999999999997e-05 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463518+0000", 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7220999999999997e-05 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463584+0000", 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229000000000006e-05 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.167 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:195215 4.15 4:ab483f24:::rbd_data.11b55ebb7981.000000000000000a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 37.959692371999999, 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015135688, 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 195215 2024-09-16T11:16:03.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:03.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086318+0000", 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.598e-06 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086326+0000", 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0180000000000001e-06 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086347+0000", 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0222000000000001e-05 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:03.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086359+0000", 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2155e-05 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086451+0000", 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1815000000000001e-05 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086557+0000", 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010589 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.088907+0000", 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002350651 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101185+0000", 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01227816 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101257+0000", 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1883000000000003e-05 2024-09-16T11:16:03.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101359+0000", 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101766 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101453+0000", 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3529999999999994e-05 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:03.173 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:03.174 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:03.175 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:03.175 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:03.175 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:03.176 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:03.169+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:03.176 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:03.181 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:03.204 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:03.227 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:03.249 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:03.271 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:16:03.273 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:03.273 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:03.273 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:03.273 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:03.273 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:03.274 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:03.274 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:03.274 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:16:03.275 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:03.275 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:03.275 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:03.272+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:03.284 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:03.376 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:03.369+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:03.431 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:03.431 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:03.440 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:03.478 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:03.476+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:03.539 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:03.539 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:03.548 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:03.579 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:03.576+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:03.603 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4] out_osds: [5, 7, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:16:03.604 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:16:03.604 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T11:16:03.604 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T11:16:03.628 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:16:03.673 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:03.673 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:03.680 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:03.673+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:03.687 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:03.781 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:03.776+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:03.824 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:03.824 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:03.832 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:03.882 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:03.877+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout: "epoch": 176, 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout: "fsid": "7e3ff0c6-8090-4b11-95d6-fd611813de49", 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout: "created": "2024-09-16T11:12:35.459871+0000", 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout: "modified": "2024-09-16T11:16:03.604498+0000", 2024-09-16T11:16:03.903 INFO:teuthology.orchestra.run.smithi162.stdout: "last_up_change": "2024-09-16T11:12:46.645944+0000", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "last_in_change": "2024-09-16T11:15:44.487729+0000", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_num": 5801984, 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_set": [ 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "noscrub", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "pglog_hardlimit", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "purged_snapdirs", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_deletes", 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "sortbitwise" 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_version": 4, 2024-09-16T11:16:03.904 INFO:teuthology.orchestra.run.smithi162.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_snapshot": "", 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_max": 4, 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "max_osd": 8, 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "require_min_compat_client": "luminous", 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "min_compat_client": "jewel", 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "require_osd_release": "quincy", 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "pools": [ 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 1, 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": ".mgr", 2024-09-16T11:16:03.905 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:48.004741+0000", 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 1, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool", 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:16:03.906 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 1, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 1, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 1, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 1, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 1, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.907 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "17", 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 0, 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 0, 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:16:03.908 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:16:03.909 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:16:03.910 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "options": { 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_max": 32, 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_min": 1 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "mgr": {} 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 2, 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "rbd", 2024-09-16T11:16:03.911 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:12:53.361644+0000", 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 8, 2024-09-16T11:16:03.912 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 8, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 8, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 8, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 8, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "26", 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:16:03.913 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [ 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: 3 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:16:03.914 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": 3, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": 3, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:16:03.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:16:03.916 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 3, 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "cache", 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:02.524586+0000", 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8201, 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,incomplete_clones,selfmanaged_snaps", 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:16:03.917 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 4, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 4, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 4, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 4, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 4, 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:16:03.918 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "30", 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "26", 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "26", 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "26", 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 3, 2024-09-16T11:16:03.919 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 24, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": 2, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "writeback", 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 250, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:16:03.920 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "bloom", 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "false_positive_probability": 0.050000000000000003, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "target_size": 0, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "seed": 0 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 60, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 8, 2024-09-16T11:16:03.921 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [ 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000, 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: 1000000 2024-09-16T11:16:03.922 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_name": "pool_client.0", 2024-09-16T11:16:03.923 INFO:teuthology.orchestra.run.smithi162.stdout: "create_time": "2024-09-16T11:13:11.876887+0000", 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "flags": 8193, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "type": 1, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 2, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "min_size": 1, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_rule": 0, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_count": 0, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_target": 0, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "object_hash": 2, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_autoscale_mode": "off", 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num": 32, 2024-09-16T11:16:03.924 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num": 32, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_placement_num_target": 32, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_target": 32, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_num_pending": 32, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "last_pg_merge_meta": { 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "source_pgid": "0.0", 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "ready_epoch": 0, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_started": 0, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "last_epoch_clean": 0, 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "source_version": "0'0", 2024-09-16T11:16:03.925 INFO:teuthology.orchestra.run.smithi162.stdout: "target_version": "0'0" 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "last_change": "176", 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend": "0", 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "auid": 0, 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_mode": "selfmanaged", 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_seq": 66, 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "snap_epoch": 176, 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "pool_snaps": [], 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps": "[]", 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_bytes": 0, 2024-09-16T11:16:03.926 INFO:teuthology.orchestra.run.smithi162.stdout: "quota_max_objects": 0, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tiers": [], 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "tier_of": -1, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "read_tier": -1, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "write_tier": -1, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_mode": "none", 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_bytes": 0, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "target_max_objects": 0, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_flush_age": 0, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "cache_min_evict_age": 0, 2024-09-16T11:16:03.927 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profile": "", 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_params": { 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "none" 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_period": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_count": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "use_gmt_hitset": true, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "min_read_recency_for_promote": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "min_write_recency_for_promote": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "hit_set_search_last_n": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "grade_table": [], 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "stripe_width": 0, 2024-09-16T11:16:03.928 INFO:teuthology.orchestra.run.smithi162.stdout: "expected_num_objects": 0, 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "fast_read": false, 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "options": {}, 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "application_metadata": { 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "rbd": {} 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "osds": [ 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "b772f030-6c52-46c6-a992-36cadd7f9edd", 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:16:03.929 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 174, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6824", 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6825", 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6826", 2024-09-16T11:16:03.931 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6827", 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.932 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6830", 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6831", 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.933 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6828", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6829", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22248 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6825/22248", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6827/22248", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6831/22248", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6829/22248", 2024-09-16T11:16:03.934 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.935 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.935 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.935 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.936 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:16:03.936 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "6335aa8f-a82b-45a4-be04-b1b064314f8d", 2024-09-16T11:16:03.936 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.937 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 0, 2024-09-16T11:16:03.937 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 0, 2024-09-16T11:16:03.937 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 0, 2024-09-16T11:16:03.937 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.937 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.938 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.938 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 140, 2024-09-16T11:16:03.938 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.938 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.938 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.939 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.939 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6800", 2024-09-16T11:16:03.939 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.940 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.940 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6801", 2024-09-16T11:16:03.940 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.940 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.941 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.941 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.942 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6802", 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6803", 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6806", 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6807", 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.944 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6804", 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.945 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6805", 2024-09-16T11:16:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22247 2024-09-16T11:16:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6801/22247", 2024-09-16T11:16:03.946 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6803/22247", 2024-09-16T11:16:03.947 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6807/22247", 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6805/22247", 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "8d9432da-1713-44e1-9c96-db33f21cf333", 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.948 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 172, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6816", 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6817", 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6818", 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6819", 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6822", 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6823", 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6820", 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6821", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22258 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6817/22258", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6819/22258", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6823/22258", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6821/22258", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.953 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "f17d5292-6b43-4887-a146-a647c60e4f86", 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 0.7781982421875, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 162, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.954 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6808", 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6809", 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.955 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6810", 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6811", 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6814", 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6815", 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.957 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6812", 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.162:6813", 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22257 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.958 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.162:6809/22257", 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.162:6811/22257", 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.162:6815/22257", 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.162:6813/22257", 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "93cb4050-ba1d-44c4-8e05-854a4cbb9236", 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:16:03.959 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 174, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6800", 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6801", 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6802", 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6803", 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6806", 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6807", 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6804", 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6805", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22379 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6801/22379", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6803/22379", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6807/22379", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6805/22379", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.964 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "432a0446-4f3b-436d-bba0-76184260e13c", 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 0, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 0, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 90, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.965 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6808", 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6809", 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.966 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6810", 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6811", 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6814", 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6815", 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.968 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6812", 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6813", 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22380 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6809/22380", 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6811/22380", 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6815/22380", 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6813/22380", 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "7c7e65e2-ab06-47f4-8d6d-712aa308e13f", 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.970 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 1, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 1, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 161, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6816", 2024-09-16T11:16:03.971 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6818", 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.972 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6819", 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6821", 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6826", 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6827", 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.974 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6822", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6824", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22381 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6818/22381", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6821/22381", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6827/22381", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6824/22381", 2024-09-16T11:16:03.975 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "uuid": "827daf60-a01e-40a5-9595-7ec5406e4a8a", 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "up": 1, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "in": 0, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "weight": 0, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_affinity": 1, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_begin": 0, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "last_clean_end": 0, 2024-09-16T11:16:03.976 INFO:teuthology.orchestra.run.smithi162.stdout: "up_from": 13, 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "up_thru": 113, 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "down_at": 0, 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "lost_at": 0, 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addrs": { 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6817", 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6820", 2024-09-16T11:16:03.977 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addrs": { 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6823", 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6825", 2024-09-16T11:16:03.978 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addrs": { 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6830", 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6831", 2024-09-16T11:16:03.979 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addrs": { 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "addrvec": [ 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v2", 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6828", 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "type": "v1", 2024-09-16T11:16:03.980 INFO:teuthology.orchestra.run.smithi162.stdout: "addr": "172.21.15.165:6829", 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "nonce": 22378 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "public_addr": "172.21.15.165:6820/22378", 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "cluster_addr": "172.21.15.165:6825/22378", 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_back_addr": "172.21.15.165:6831/22378", 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "heartbeat_front_addr": "172.21.15.165:6829/22378", 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "state": [ 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "exists", 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: "up" 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.981 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_xinfo": [ 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 0, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.054827+0000", 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 1, 2024-09-16T11:16:03.982 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 65536, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.474331+0000", 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 2, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.983 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.753279+0000", 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 3, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.733978+0000", 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 4, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.980000+0000", 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 5, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.985 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 65536, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.142568+0000", 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 6, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 0, 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:42.883591+0000", 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "osd": 7, 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "down_stamp": "0.000000", 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_probability": 0, 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "laggy_interval": 0, 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "features": 4540138320759226367, 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "old_weight": 65536, 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "last_purged_snaps_scrub": "2024-09-16T11:12:43.129808+0000", 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "dead_epoch": 0 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap": [], 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_upmap_items": [], 2024-09-16T11:16:03.987 INFO:teuthology.orchestra.run.smithi162.stdout: "pg_temp": [], 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "primary_temp": [], 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "blocklist": {}, 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "range_blocklist": {}, 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "erasure_code_profiles": { 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "default": { 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "crush-failure-domain": "osd", 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "k": "2", 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "m": "1", 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "plugin": "jerasure", 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "technique": "reed_sol_van" 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: "removed_snaps_queue": [ 2024-09-16T11:16:03.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "snaps": [ 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "begin": 25, 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "length": 5 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "new_removed_snaps": [ 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "pool": 4, 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: "snaps": [ 2024-09-16T11:16:03.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "begin": 25, 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "length": 1 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: ], 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "new_purged_snaps": [], 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "crush_node_flags": {}, 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "device_class_flags": {}, 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode": { 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_enabled": false, 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_bucket_count": 0, 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "degraded_stretch_mode": 0, 2024-09-16T11:16:03.990 INFO:teuthology.orchestra.run.smithi162.stdout: "recovering_stretch_mode": 0, 2024-09-16T11:16:03.991 INFO:teuthology.orchestra.run.smithi162.stdout: "stretch_mode_bucket": 0 2024-09-16T11:16:03.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:03.991 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:03.993 INFO:tasks.thrashosds.thrasher:j is {'epoch': 176, 'fsid': '7e3ff0c6-8090-4b11-95d6-fd611813de49', 'created': '2024-09-16T11:12:35.459871+0000', 'modified': '2024-09-16T11:16:03.604498+0000', 'last_up_change': '2024-09-16T11:12:46.645944+0000', 'last_in_change': '2024-09-16T11:15:44.487729+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T11:12:48.004741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T11:12:53.361644+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-09-16T11:13:02.524586+0000', 'flags': 8201, 'flags_names': 'hashpspool,incomplete_clones,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '26', 'last_force_op_resend_prenautilus': '26', 'last_force_op_resend_preluminous': '26', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 24, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 2, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-09-16T11:13:11.876887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 66, 'snap_epoch': 176, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': 'b772f030-6c52-46c6-a992-36cadd7f9edd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 174, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6824', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6825', 'nonce': 22248}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6826', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6827', 'nonce': 22248}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6830', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6831', 'nonce': 22248}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6828', 'nonce': 22248}, {'type': 'v1', 'addr': '172.21.15.162:6829', 'nonce': 22248}]}, 'public_addr': '172.21.15.162:6825/22248', 'cluster_addr': '172.21.15.162:6827/22248', 'heartbeat_back_addr': '172.21.15.162:6831/22248', 'heartbeat_front_addr': '172.21.15.162:6829/22248', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '6335aa8f-a82b-45a4-be04-b1b064314f8d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 140, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6800', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6801', 'nonce': 22247}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6802', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6803', 'nonce': 22247}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6806', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6807', 'nonce': 22247}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6804', 'nonce': 22247}, {'type': 'v1', 'addr': '172.21.15.162:6805', 'nonce': 22247}]}, 'public_addr': '172.21.15.162:6801/22247', 'cluster_addr': '172.21.15.162:6803/22247', 'heartbeat_back_addr': '172.21.15.162:6807/22247', 'heartbeat_front_addr': '172.21.15.162:6805/22247', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8d9432da-1713-44e1-9c96-db33f21cf333', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6816', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6817', 'nonce': 22258}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6818', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6819', 'nonce': 22258}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6822', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6823', 'nonce': 22258}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6820', 'nonce': 22258}, {'type': 'v1', 'addr': '172.21.15.162:6821', 'nonce': 22258}]}, 'public_addr': '172.21.15.162:6817/22258', 'cluster_addr': '172.21.15.162:6819/22258', 'heartbeat_back_addr': '172.21.15.162:6823/22258', 'heartbeat_front_addr': '172.21.15.162:6821/22258', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f17d5292-6b43-4887-a146-a647c60e4f86', 'up': 1, 'in': 1, 'weight': 0.7781982421875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 162, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6808', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6809', 'nonce': 22257}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6810', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6811', 'nonce': 22257}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6814', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6815', 'nonce': 22257}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6812', 'nonce': 22257}, {'type': 'v1', 'addr': '172.21.15.162:6813', 'nonce': 22257}]}, 'public_addr': '172.21.15.162:6809/22257', 'cluster_addr': '172.21.15.162:6811/22257', 'heartbeat_back_addr': '172.21.15.162:6815/22257', 'heartbeat_front_addr': '172.21.15.162:6813/22257', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '93cb4050-ba1d-44c4-8e05-854a4cbb9236', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 174, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6800', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6801', 'nonce': 22379}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6802', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6803', 'nonce': 22379}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6806', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6807', 'nonce': 22379}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6804', 'nonce': 22379}, {'type': 'v1', 'addr': '172.21.15.165:6805', 'nonce': 22379}]}, 'public_addr': '172.21.15.165:6801/22379', 'cluster_addr': '172.21.15.165:6803/22379', 'heartbeat_back_addr': '172.21.15.165:6807/22379', 'heartbeat_front_addr': '172.21.15.165:6805/22379', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '432a0446-4f3b-436d-bba0-76184260e13c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6808', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6809', 'nonce': 22380}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6810', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6811', 'nonce': 22380}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6814', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6815', 'nonce': 22380}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6812', 'nonce': 22380}, {'type': 'v1', 'addr': '172.21.15.165:6813', 'nonce': 22380}]}, 'public_addr': '172.21.15.165:6809/22380', 'cluster_addr': '172.21.15.165:6811/22380', 'heartbeat_back_addr': '172.21.15.165:6815/22380', 'heartbeat_front_addr': '172.21.15.165:6813/22380', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c7e65e2-ab06-47f4-8d6d-712aa308e13f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 161, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6816', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6818', 'nonce': 22381}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6819', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6821', 'nonce': 22381}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6826', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6827', 'nonce': 22381}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6822', 'nonce': 22381}, {'type': 'v1', 'addr': '172.21.15.165:6824', 'nonce': 22381}]}, 'public_addr': '172.21.15.165:6818/22381', 'cluster_addr': '172.21.15.165:6821/22381', 'heartbeat_back_addr': '172.21.15.165:6827/22381', 'heartbeat_front_addr': '172.21.15.165:6824/22381', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '827daf60-a01e-40a5-9595-7ec5406e4a8a', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6817', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6820', 'nonce': 22378}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6823', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6825', 'nonce': 22378}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6830', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6831', 'nonce': 22378}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6828', 'nonce': 22378}, {'type': 'v1', 'addr': '172.21.15.165:6829', 'nonce': 22378}]}, 'public_addr': '172.21.15.165:6820/22378', 'cluster_addr': '172.21.15.165:6825/22378', 'heartbeat_back_addr': '172.21.15.165:6831/22378', 'heartbeat_front_addr': '172.21.15.165:6829/22378', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.054827+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.474331+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.753279+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.733978+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.980000+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.142568+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T11:12:42.883591+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-16T11:12:43.129808+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 25, 'length': 5}]}], 'new_removed_snaps': [{'pool': 4, 'snaps': [{'begin': 25, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T11:16:03.993 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T11:16:03.994 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:03.994 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:03.994 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:03.995 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:03.985+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:04.093 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:04.089+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:04.139 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.140 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:04.194 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:04.189+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:04.248 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.249 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:04.294 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:04.289+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:04.357 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.358 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.367 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:04.410 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:04.405+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:04.496 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:04.489+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:04.515 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.515 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.524 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:04.597 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:04.589+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:04.633 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:16:04.641 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.641 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:04.697 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:04.692+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:04.756 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.756 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:04.814 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:04.812+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:04.866 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.866 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.876 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:04.899 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:04.896+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:04.971 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:04.971 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:04.980 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:04.999 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:04.996+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:05.083 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:05.083 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:05.092 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:05.134 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:05.132+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:05.201 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:05.196+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:05.236 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:05.237 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:05.246 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:05.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:05.330 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:05.325+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:05.347 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:05.347 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:05.356 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:05.402 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:05.400+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:05.427 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:05.460 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:05.467 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:05.467 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:05.476 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:05.482 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:05.501 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:05.503 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:05.500+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:05.526 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:05.549 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:05.576 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:05.576 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:05.579 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:05.588 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:05.599 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:16:05.601 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:05.601 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:05.601 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:05.601 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:05.602 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:05.602 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:05.602 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:05.602 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:16:05.604 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:05.597+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:05.705 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:05.701+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:05.739 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:05.739 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:05.739 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:05.740 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:05.740 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:05.748 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:05.806 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:05.801+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:05.847 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 137.64611683199999, 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:05.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:05.852 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 137.646100765, 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:05.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 137.64608437000001, 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.857 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 137.64607479599999, 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.861 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:05.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 137.64606526, 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.865 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 108.94822837700001, 2024-09-16T11:16:05.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:05.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:05.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 94.262393122999995, 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:05.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327873779999997, 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:05.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327793108999998, 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:05.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:05.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:05.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327671717000001, 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.891 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:05.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:05.895 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327619523000003, 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:05.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327562790000002, 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.901 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.906 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327515591000001, 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.911 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327465621000002, 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:05.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.327398506999998, 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:05.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:05.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.325423420999996, 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.922 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:05.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:05.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.325374936000003, 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.927 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:05.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.325257907999998, 2024-09-16T11:16:05.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:05.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:05.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:05.937 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 60.325166733000003, 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:05.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:05.942 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 33.084759886000001, 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:05.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:05.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:05.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:05.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:05.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:05.949 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:05.951 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:16:05.951 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:05.945+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:06.052 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:06.045+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:06.061 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:06.061 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:06.061 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:06.061 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:06.070 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T11:16:06.152 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:06.148+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:06.172 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:06.172 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:06.172 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:06.172 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:06.172 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:06.181 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T11:16:06.253 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:06.248+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:06.287 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:06.287 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:06.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.50670239199999, 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13249346100000001, 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 1 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776020+0000", 2024-09-16T11:16:06.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776022+0000", 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2639999999999998e-06 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776031+0000", 2024-09-16T11:16:06.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3780000000000007e-06 2024-09-16T11:16:06.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776049+0000", 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7614000000000001e-05 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776061+0000", 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1505000000000001e-05 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776147+0000", 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6657999999999996e-05 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.776157+0000", 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9969999999999993e-06 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905612+0000", 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.129455229 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.905933+0000", 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000320649 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.906219+0000", 2024-09-16T11:16:06.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000285524 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.907972+0000", 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001753498 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908335+0000", 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036296899999999999 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908450+0000", 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115294 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:49.908513+0000", 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2881999999999999e-05 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11877669200001, 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013122781, 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.297 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 44 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163946+0000", 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.163947+0000", 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.517e-06 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164064+0000", 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000117401 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164079+0000", 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4333e-05 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164086+0000", 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0659999999999998e-06 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170372+0000", 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0062863279999999999 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170455+0000", 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2534999999999995e-05 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.170557+0000", 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102468 2024-09-16T11:16:06.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173417+0000", 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0028598230000000001 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176972+0000", 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035547920000000002 2024-09-16T11:16:06.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177024+0000", 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2652999999999999e-05 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177068+0000", 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3865000000000001e-05 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.303 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11862536199999, 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013166041999999999, 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.304 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 45 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164097+0000", 2024-09-16T11:16:06.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164098+0000", 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0270000000000001e-06 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164104+0000", 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.75e-06 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164132+0000", 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8296000000000001e-05 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164138+0000", 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.4930000000000003e-06 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171130+0000", 2024-09-16T11:16:06.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0069912430000000003 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171216+0000", 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6363e-05 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171338+0000", 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000121922 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173827+0000", 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024891340000000001 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177179+0000", 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033522489999999999 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177223+0000", 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3380999999999997e-05 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177263+0000", 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0184e-05 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.118573876, 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013287731000000001, 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 46 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164148+0000", 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164149+0000", 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0999999999999997e-07 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164154+0000", 2024-09-16T11:16:06.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.95e-06 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164190+0000", 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5621999999999999e-05 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164206+0000", 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5930000000000002e-05 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171784+0000", 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0075779699999999998 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171841+0000", 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7222000000000002e-05 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.171933+0000", 2024-09-16T11:16:06.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1955999999999998e-05 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173837+0000", 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0019043689999999999 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177355+0000", 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0035179959999999998 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177407+0000", 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2200999999999997e-05 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177436+0000", 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8705e-05 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.317 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11850372800001, 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013366232, 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 47 2024-09-16T11:16:06.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.319 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164219+0000", 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5700000000000001e-07 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164352+0000", 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013254400000000001 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164373+0000", 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1081999999999999e-05 2024-09-16T11:16:06.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164382+0000", 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3600000000000002e-06 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172387+0000", 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0080049909999999995 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172459+0000", 2024-09-16T11:16:06.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1211000000000006e-05 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.172585+0000", 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012662099999999999 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176907+0000", 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0043213790000000002 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177511+0000", 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00060465300000000003 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177550+0000", 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8504000000000003e-05 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177585+0000", 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5030000000000002e-05 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.323 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11833125800001, 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013340171, 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48 2024-09-16T11:16:06.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164391+0000", 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164392+0000", 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8000000000000004e-07 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164397+0000", 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3229999999999997e-06 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164414+0000", 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6523000000000001e-05 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164419+0000", 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6330000000000001e-06 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173876+0000", 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.009456585 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.173975+0000", 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.9159000000000004e-05 2024-09-16T11:16:06.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174127+0000", 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000151865 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176921+0000", 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0027938709999999999 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177656+0000", 2024-09-16T11:16:06.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000734988 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177695+0000", 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9502000000000002e-05 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177731+0000", 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5842e-05 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11827293600001, 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013446415999999999, 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 49 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164449+0000", 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164450+0000", 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8999999999999995e-07 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164455+0000", 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9599999999999999e-06 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164476+0000", 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0863e-05 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164484+0000", 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6159999999999996e-06 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174621+0000", 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010137006 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174728+0000", 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107113 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.174863+0000", 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013541200000000001 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177282+0000", 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024188399999999998 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177824+0000", 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00054216500000000005 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177863+0000", 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9053e-05 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177896+0000", 2024-09-16T11:16:06.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2598000000000002e-05 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11812081599999, 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.013960505999999999, 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164601+0000", 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164603+0000", 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2729999999999999e-06 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164754+0000", 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015102 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164765+0000", 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1046999999999999e-05 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164772+0000", 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.776e-06 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175361+0000", 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010589777999999999 2024-09-16T11:16:06.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175430+0000", 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8915e-05 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.175556+0000", 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000126076 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177303+0000", 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00174672 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178484+0000", 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001181388 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178527+0000", 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2619e-05 2024-09-16T11:16:06.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178562+0000", 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4894000000000002e-05 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11794229700001, 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015932242999999999, 2024-09-16T11:16:06.342 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 51 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164780+0000", 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164781+0000", 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2699999999999998e-07 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164786+0000", 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960000000000004e-06 2024-09-16T11:16:06.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164800+0000", 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3864e-05 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164806+0000", 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7699999999999998e-06 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176280+0000", 2024-09-16T11:16:06.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011474061000000001 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176349+0000", 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9210999999999999e-05 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.176464+0000", 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000114808 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177914+0000", 2024-09-16T11:16:06.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001450326 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180643+0000", 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002729052 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180682+0000", 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8346000000000002e-05 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.180712+0000", 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0382000000000002e-05 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.11790932400001, 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.016485116000000001, 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.348 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 52 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164813+0000", 2024-09-16T11:16:06.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164814+0000", 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9999999999999996e-07 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164819+0000", 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6230000000000002e-06 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164828+0000", 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2560000000000002e-06 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.164834+0000", 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6839999999999999e-06 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.177952+0000", 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01311728 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178029+0000", 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7588000000000003e-05 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.178154+0000", 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124475 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.179140+0000", 2024-09-16T11:16:06.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00098608700000000007 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181207+0000", 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002067686 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181262+0000", 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4891e-05 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.181298+0000", 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5746e-05 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 195.565721599, 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014493858, 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 72 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717001+0000", 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2600000000000002e-07 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717104+0000", 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010329 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717117+0000", 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.22e-05 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717123+0000", 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.489e-06 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726216+0000", 2024-09-16T11:16:06.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090927109999999998 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726300+0000", 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4376000000000007e-05 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.726521+0000", 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000220502 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731198+0000", 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0046769059999999998 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731300+0000", 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010231899999999999 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731410+0000", 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000109701 2024-09-16T11:16:06.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731494+0000", 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4838000000000003e-05 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:16:06.360 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 195.565592802, 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014509454999999999, 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 73 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717129+0000", 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717130+0000", 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5000000000000002e-07 2024-09-16T11:16:06.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717140+0000", 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5959999999999993e-06 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717170+0000", 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0463e-05 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717182+0000", 2024-09-16T11:16:06.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1982e-05 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727604+0000", 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010421975 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727737+0000", 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013290099999999999 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.727889+0000", 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015185599999999999 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731203+0000", 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033140370000000001 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731546+0000", 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000342801 2024-09-16T11:16:06.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731599+0000", 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3202999999999999e-05 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731639+0000", 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9990999999999997e-05 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 195.56553445700001, 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014802111999999999, 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 74 2024-09-16T11:16:06.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717188+0000", 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2200000000000004e-07 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717192+0000", 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8360000000000002e-06 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717200+0000", 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5259999999999996e-06 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717203+0000", 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7100000000000001e-06 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729013+0000", 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011809887 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729075+0000", 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2093000000000003e-05 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729271+0000", 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00019512199999999999 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731700+0000", 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0024296909999999999 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731808+0000", 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108177 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731912+0000", 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000103563 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.731990+0000", 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7885000000000004e-05 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 195.565480068, 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015733326999999998, 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 75 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717242+0000", 2024-09-16T11:16:06.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717243+0000", 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5000000000000002e-07 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717352+0000", 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010889499999999999 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717364+0000", 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1892000000000001e-05 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717370+0000", 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2899999999999999e-06 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729881+0000", 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012511216 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.729994+0000", 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113124 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.730210+0000", 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021523500000000001 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732718+0000", 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002508447 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732788+0000", 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0047000000000002e-05 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732890+0000", 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102321 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732975+0000", 2024-09-16T11:16:06.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.5110000000000003e-05 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 195.565345761, 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.017216540999999998, 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 76 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717376+0000", 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717377+0000", 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8400000000000004e-07 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717381+0000", 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.014e-06 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717396+0000", 2024-09-16T11:16:06.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4897e-05 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.717401+0000", 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.164e-06 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732062+0000", 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014661196 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732141+0000", 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8257999999999995e-05 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.732267+0000", 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012650699999999999 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.733629+0000", 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0013615680000000001 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734466+0000", 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00083781100000000005 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734556+0000", 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9290999999999996e-05 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:50.734593+0000", 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7251e-05 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.384 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 194.78410307999999, 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.012293635000000001, 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 93 2024-09-16T11:16:06.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498619+0000", 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498620+0000", 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.31e-06 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498792+0000", 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017158599999999999 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498821+0000", 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9436000000000001e-05 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498839+0000", 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7975999999999999e-05 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506636+0000", 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0077969770000000001 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.506796+0000", 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015933200000000001 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.507038+0000", 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024176199999999999 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510800+0000", 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003762675 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510810+0000", 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5559999999999996e-06 2024-09-16T11:16:06.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510865+0000", 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4968e-05 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.510913+0000", 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8056999999999997e-05 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4203.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 194.78386495199999, 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01280746, 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4203", 2024-09-16T11:16:06.391 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/622243646", 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 94 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498857+0000", 2024-09-16T11:16:06.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498859+0000", 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6929999999999999e-06 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498868+0000", 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4229999999999994e-06 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498899+0000", 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0660000000000001e-05 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.498926+0000", 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7402e-05 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509177+0000", 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.010250296000000001 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509365+0000", 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000188443 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.509599+0000", 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000233704 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511563+0000", 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001964379 2024-09-16T11:16:06.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511570+0000", 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0870000000000001e-06 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511627+0000", 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.6734000000000003e-05 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:12:51.511665+0000", 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.7639000000000003e-05 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94661 4.11 e76/31 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head v 76'349, mlcod=74'347)", 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.39529250699999, 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.148638618, 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887430+0000", 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.887432+0000", 2024-09-16T11:16:06.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.322e-06 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924473+0000", 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.037040854999999998 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924505+0000", 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.1868e-05 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924518+0000", 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3447e-05 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924611+0000", 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2689999999999995e-05 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.924658+0000", 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7233e-05 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.035972+0000", 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.11131360699999999 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.036068+0000", 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6595999999999995e-05 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.401 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:194870 4.8 4:13a4eb47:::rbd_data.11b55ebb7981.00000000000000cd:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 41.834198997000001, 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015060858999999999, 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 194870 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448523+0000", 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448525+0000", 2024-09-16T11:16:06.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.187e-06 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448539+0000", 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3705e-05 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448566+0000", 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6937e-05 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448576+0000", 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0032e-05 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448682+0000", 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106353 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.448818+0000", 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000135308 2024-09-16T11:16:06.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.452297+0000", 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0034793049999999998 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463364+0000", 2024-09-16T11:16:06.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.011067184000000001 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463431+0000", 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6397999999999997e-05 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463518+0000", 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7220999999999997e-05 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:24.463584+0000", 2024-09-16T11:16:06.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6229000000000006e-05 2024-09-16T11:16:06.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.409 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.409 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:195215 4.15 4:ab483f24:::rbd_data.11b55ebb7981.000000000000000a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138)", 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 41.196405396999999, 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.015135688, 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.410 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 195215 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086317+0000", 2024-09-16T11:16:06.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086318+0000", 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.598e-06 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086326+0000", 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0180000000000001e-06 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.412 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086347+0000", 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0222000000000001e-05 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086359+0000", 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2155e-05 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086451+0000", 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1815000000000001e-05 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.086557+0000", 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010589 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.088907+0000", 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002350651 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101185+0000", 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.01227816 2024-09-16T11:16:06.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101257+0000", 2024-09-16T11:16:06.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1883000000000003e-05 2024-09-16T11:16:06.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101359+0000", 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000101766 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:25.101453+0000", 2024-09-16T11:16:06.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3529999999999994e-05 2024-09-16T11:16:06.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.418 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.418 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:06.418 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:06.418 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:06.419 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:06.417+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:06.420 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:06.521 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:06.513+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:06.530 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:06.530 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:06.530 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:06.530 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:06.539 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:06.621 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:06.617+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:06.642 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:06.642 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:06.643 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:06.648 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:16:06.652 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:06.723 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:06.720+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.61439319600001, 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:16:06.793 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:16:06.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.61072168699999, 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.798 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:16:06.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:06.802 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.608656395, 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:16:06.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.60864253899999, 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:16:06.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:16:06.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.60862866400001, 2024-09-16T11:16:06.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:16:06.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.608614897, 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.60856738800001, 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.822 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:16:06.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:16:06.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.60828157, 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.827 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:16:06.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:16:06.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.515617754, 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:16:06.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:16:06.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:16:06.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.515468572, 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.838 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:16:06.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.843 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.527067352000003, 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.844 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:16:06.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.527020391999997, 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.849 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:16:06.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:16:06.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.5270017, 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:06.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:16:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:16:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.526985500999999, 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.859 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:16:06.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.526970458999998, 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:16:06.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:06.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:16:06.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.526954709000002, 2024-09-16T11:16:06.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:16:06.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:16:06.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.526939419000001, 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:16:06.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:16:06.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.526923840999999, 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.879 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.526908861999999, 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.884 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:06.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:16:06.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.888 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 34.031419110000002, 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.889 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:16:06.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:06.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:06.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:06.895 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:06.897 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:06.897 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:06.889+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:06.997 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:06.997 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:06.998 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:06.998 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:06.998 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:06.993+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:07.006 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:07.099 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:07.093+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:07.101 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:07.101 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:07.101 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:07.101 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:07.101 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:07.110 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:07.200 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:07.196+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:07.212 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.715246521, 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.213 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:07.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:16:07.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:07.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.461833723, 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:16:07.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.461564407, 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.225 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:16:07.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:16:07.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.461165897, 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.231 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:07.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.46087710899999, 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:16:07.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:16:07.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.459177693, 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:16:07.240 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:16:07.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:16:07.244 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.459099377, 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.245 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:16:07.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:16:07.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:07.249 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.459065638, 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:16:07.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:16:07.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.45902489300001, 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:16:07.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.455824445, 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.258 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:07.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.260 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.262 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.455806388, 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:07.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.266 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.45578039900001, 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:16:07.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:16:07.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.455081968, 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:16:07.271 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:16:07.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:16:07.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.455069409, 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:16:07.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:16:07.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 72.545393656000002, 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:16:07.280 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:16:07.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:16:07.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.862968150999997, 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.288 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:16:07.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:07.294 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.860103258000002, 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:16:07.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:16:07.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.301 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.851057341999997, 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:16:07.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:16:07.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.793641837000003, 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.308 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:16:07.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:16:07.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:16:07.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.747382672000001, 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:07.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:16:07.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.321 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:07.322 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:07.323 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:07.320+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:07.423 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:07.420+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:07.425 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:07.425 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:07.425 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:07.435 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:07.524 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:07.517+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:07.533 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:07.533 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:07.533 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:07.533 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:07.533 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:07.542 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:07.612 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:07.625 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:07.620+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:07.653 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.57166248, 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:07.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.57164802200001, 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:07.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571630822, 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:07.664 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.57161730200001, 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:07.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:07.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.57160876, 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:07.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:07.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:07.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571598616, 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.680 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571585109, 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:07.685 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.690 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571571517, 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.692 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:07.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:07.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.696 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571547235, 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:07.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:07.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.57151307399999, 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.703 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:07.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571479677, 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.708 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:07.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:07.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571457032, 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:07.713 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:07.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:07.718 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.57144145700001, 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:07.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.571424423, 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.724 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:07.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:07.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.56779145900001, 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.729 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:07.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.567757117, 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.734 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:07.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.737 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.56774117099999, 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.739 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:07.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.56772556, 2024-09-16T11:16:07.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:07.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.746 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:07.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:07.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.749 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.567710472, 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.750 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.751 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:07.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 105.56769567000001, 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:07.755 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:07.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:07.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:07.760 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:07.761 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:16:07.762 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:07.763 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:07.784 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:07.784 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:07.777+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:07.805 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:07.823 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:07.844 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:07.865 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:07.861+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:07.866 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:07.879 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:07.879 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:07.886 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:07.887 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:07.914 INFO:teuthology.orchestra.run.smithi162.stdout:osd.7: {} 2024-09-16T11:16:07.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:07.916 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:07.917 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:07.917 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:07.917 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:07.917 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:07.917 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:07.917 INFO:teuthology.orchestra.run.smithi162.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T11:16:07.970 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:07.965+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:08.008 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.008 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.017 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:08.068 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:08.061+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:08.118 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.118 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.128 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:08.170 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:08.165+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:08.235 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.235 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.244 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:08.270 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:08.265+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:08.342 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.342 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.351 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:08.371 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:08.368+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:08.451 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.452 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.461 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:08.472 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:08.468+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:08.563 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.563 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.572 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:08.573 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:08.569+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:08.674 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:16:08.672+0000 7fa4f481f700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22375) UID: 0 2024-09-16T11:16:08.736 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.736 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.748 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:08.775 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:08.772+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:08.846 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:08.846 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:08.855 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:08.902 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:08.900+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:08.977 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:08.972+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:08.994 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4] out_osds: [5, 7, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:16:08.994 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:16:08.995 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T11:16:08.995 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T11:16:08.995 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:16:09.004 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.004 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.013 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:09.059 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T11:16:09.059 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-09-16T11:16:09.059 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:16:09.060 DEBUG:teuthology.orchestra.run.smithi165:> type ceph-objectstore-tool 2024-09-16T11:16:09.065 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T11:16:09.066 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-09-16T11:16:09.078 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:09.073+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:09.153 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.153 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.163 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:09.179 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:09.176+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:09.259 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.259 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.269 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:09.314 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:09.309+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:09.374 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.375 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.381 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:09.376+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:09.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:09.484 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:09.477+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:09.488 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.488 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.498 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:09.583 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:09.580+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:09.599 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.599 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.608 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:09.683 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:09.680+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:09.723 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.723 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.733 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:09.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:16:09.736+0000 7f2a4e5ec300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T11:16:09.764 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:16:09.809 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:09.805+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:09.838 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.838 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.849 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:09.885 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:09.880+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:09.927 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:09.946 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:09.946 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:09.955 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:10.002 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:09.997+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:10.065 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:10.083 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:10.088 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:10.084+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:10.100 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:10.107 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:10.107 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:10.107 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:10.107 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:10.116 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:10.119 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:10.139 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:10.161 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:10.182 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:10.183 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:10.186 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:10.187 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:10.188 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:10.181+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:10.209 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:10.209 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:10.209 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:10.209 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:10.209 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:10.218 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:10.289 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:10.281+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:10.313 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:10.314 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:10.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:10.314 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:10.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233551959, 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.316 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:10.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:10.321 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233537501, 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.322 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:10.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:10.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233520301, 2024-09-16T11:16:10.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.328 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:10.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:10.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233506781, 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.333 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:10.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:10.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233498239, 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.338 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:10.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:10.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233488095, 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.343 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:10.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:10.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.23347458800001, 2024-09-16T11:16:10.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:10.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.351 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.353 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.23346099600001, 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:10.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.358 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.23343671400001, 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.360 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:10.361 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:10.362 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233402553, 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:10.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:10.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.368 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.23336915599999, 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:10.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:10.373 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.23334651099999, 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.374 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.375 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:10.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:10.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.233330936, 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.379 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:10.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.383 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.384 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.23331390200001, 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.385 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:10.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:10.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:10.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.229680938, 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:10.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:10.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.22964659599999, 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.395 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:10.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:10.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.399 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.22963065, 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.400 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:10.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:10.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.406 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.229615039, 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.407 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:10.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:10.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.229599951, 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.412 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:10.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:10.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.229585149, 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:10.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.422 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:10.424 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:10.424 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:10.420+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:10.464 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:16:10.524 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:10.520+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:10.569 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:10.569 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:10.569 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:10.569 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:10.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:10.625 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:10.617+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:10.677 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:10.678 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:10.678 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:10.678 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:10.678 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:10.687 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:10.727 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:10.724+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:10.779 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.64318381300001, 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.780 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.64316542500001, 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:10.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:10.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:10.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.64286466799999, 2024-09-16T11:16:10.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.793 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.64283496300001, 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.797 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.64282156499999, 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:10.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:10.802 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.642807761, 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:10.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.642432762, 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.807 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:10.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:10.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.62888873099999, 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:10.811 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.812 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.815 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.628867157, 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:10.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:10.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.628850658, 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:10.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:10.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:10.823 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.628834582, 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.824 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.62845100800001, 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.828 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.829 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:10.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:10.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.832 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.61636063, 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.833 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.834 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.835 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:10.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.616352608, 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:10.837 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.838 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.839 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.840 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:10.841 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.93287372899999, 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:10.843 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:10.844 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:10.845 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.846 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 108.699455378, 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.847 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:10.848 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:10.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:10.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 99.313062005000006, 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:10.854 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:10.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 97.361457357000006, 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:10.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:10.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:10.866 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.225660425000001, 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:10.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 23.172936506999999, 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:10.872 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:10.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:10.877 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:10.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:10.880 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:10.882 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:10.882 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:10.877+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:10.982 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:10.977+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:10.986 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:10.986 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:10.987 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:10.987 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:10.996 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:11.083 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:11.080+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:11.088 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:11.088 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:11.088 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.097 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:11.185 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:11.177+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:11.203 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124898855, 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:11.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:11.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.208 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124858093, 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.209 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:11.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.211 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:11.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124827295, 2024-09-16T11:16:11.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:11.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:11.219 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124808707, 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:11.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.224 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124790655, 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:11.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:11.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.229 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124770728, 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:11.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.233 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.234 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.124751889, 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:11.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.239 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.12455994, 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:11.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.243 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.12454065999999, 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.245 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:11.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:11.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:11.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.12104909200001, 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.250 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:11.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:11.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:11.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.121039021, 2024-09-16T11:16:11.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:11.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:11.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.12101071799999, 2024-09-16T11:16:11.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:11.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:11.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.120996789, 2024-09-16T11:16:11.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:11.270 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.12098802600001, 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:11.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.120964207, 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:11.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:11.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.120933085, 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:11.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:11.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.12088534599999, 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.286 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:11.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 109.120863238, 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.291 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.711668873999997, 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:11.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:11.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 57.711622894000001, 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.303 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:11.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.310 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.311 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:11.312 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:11.309+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:11.405 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.405 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:11.406 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:11.406 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.412 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:11.409+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:11.414 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:11.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-09-16T11:16:11.508+0000 7f2a4e5ec300 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-09-16T11:16:11.513 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:11.509+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:11.514 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.514 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:11.514 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:11.514 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:11.514 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.523 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:11.615 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:11.612+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:11.616 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 170.749654118, 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.617 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.618 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.619 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.620 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.621 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58168246899999, 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.622 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.623 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:16:11.624 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.625 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:16:11.626 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:11.627 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58167090800001, 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.628 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:11.629 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.630 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.631 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.632 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58165770599999, 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.633 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.634 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:16:11.635 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.636 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.637 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58164526600001, 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:16:11.638 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.639 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.640 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:16:11.641 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.642 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58163175499999, 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.643 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.644 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.645 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:16:11.646 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.647 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58162084, 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.648 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.649 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.650 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:16:11.651 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.652 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58159492199999, 2024-09-16T11:16:11.653 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:11.654 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.655 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.655 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.655 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:11.655 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.655 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:16:11.656 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.657 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:16:11.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.581579468, 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.659 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:16:11.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:16:11.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58156958199999, 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.664 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.666 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:16:11.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:11.669 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58155981900001, 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:11.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:16:11.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.674 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.581549642, 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:16:11.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58153315999999, 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.680 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:16:11.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.684 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58151846300001, 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.685 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:11.686 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.687 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:16:11.689 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58150001999999, 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.690 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:16:11.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:16:11.694 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58148307100001, 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.695 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:16:11.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.699 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58146385800001, 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:16:11.700 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:11.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:16:11.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.581446642, 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.706 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:16:11.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.710 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58142538800001, 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:11.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:16:11.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.58140542000001, 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:11.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:11.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:11.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:16:11.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.721 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:11.721 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:11.721 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.723 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:11.723 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:11.720+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:11.823 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:11.820+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:11.865 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.866 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:11.866 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:11.866 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:11.924 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:11.917+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:11.978 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:11.978 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:11.978 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:11.978 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:11.978 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:11.986 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:12.024 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:12.017+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:12.081 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:12.081 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:12.081 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.87953721, 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.082 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:12.083 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.084 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:12.085 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.87952114300001, 2024-09-16T11:16:12.086 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.087 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:12.088 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.87950474799999, 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.091 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.092 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.093 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.879495174, 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.095 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 143.87948563800001, 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:12.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.101 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.102 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:12.103 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 115.181648755, 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.104 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.105 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:12.106 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:12.107 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:12.108 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 100.495813501, 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.109 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:12.110 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.111 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.112 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:12.113 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:12.114 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.561294157999995, 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.115 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.116 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:12.117 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:12.118 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.119 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.120 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.561213487000003, 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.121 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.122 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.123 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:12.124 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.125 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.561092095000006, 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.126 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.127 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:12.128 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.129 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.130 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.561039901000001, 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.131 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.132 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.133 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.134 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:12.135 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.136 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.560983168000007, 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.137 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:12.138 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.139 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:12.140 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:12.141 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.560935968999999, 2024-09-16T11:16:12.142 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.143 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:12.144 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.145 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:12.146 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.147 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.560885999000007, 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.148 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:12.149 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:12.150 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.150 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.150 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.150 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.152 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.153 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.154 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.560818885000003, 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.155 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.156 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:12.157 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.158 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.159 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.558843799000002, 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.160 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.161 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:12.162 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.163 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.164 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.165 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.558795313999994, 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.166 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.168 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.169 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:12.170 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.558678286000003, 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.171 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.172 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:12.173 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.174 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.175 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.176 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 66.558587110999994, 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.177 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.178 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.179 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:12.180 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:12.181 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 39.318180263999999, 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:12.182 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.183 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.184 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:12.185 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:12.186 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:12.187 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:12.188 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:12.189 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:12.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:12.191 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:12.188+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:12.192 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:12.291 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:12.288+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:12.300 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:12.300 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:12.300 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:12.311 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:12.330 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:12.352 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:12.369 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:12.387 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:12.392 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:12.388+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:12.404 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:12.404 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:12.407 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:12.413 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:12.436 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:12.457 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:12.458 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:12.459 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:12.461 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:12.461 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:12.480 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:16:12.507 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:12.507 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:12.507 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:12.501+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:12.517 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:12.594 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:12.592+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:12.677 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:12.677 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:12.690 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:12.695 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:12.689+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:12.784 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:12.784 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:12.793 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:12.796 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:12.792+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:12.862 INFO:tasks.thrashosds.thrasher:No PGs found for osd.7 2024-09-16T11:16:12.887 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:12.887 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:12.896 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:12.906 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:12.904+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:12.994 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:12.995 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:12.998 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:12.993+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:13.004 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:13.099 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:13.096+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:13.101 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.101 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.110 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:13.200 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:13.193+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:13.209 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.209 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.219 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:13.301 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:13.296+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:13.361 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.362 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.371 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:13.403 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:13.397+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:13.467 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.468 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.480 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:13.484 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:16:13.541 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:13.537+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:13.594 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.594 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.605 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:13.605 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:13.600+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:13.705 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:13.701+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:13.708 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.708 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.717 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:13.806 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:13.804+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:13.820 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.820 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.829 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:13.907 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:13.904+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:13.926 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:13.926 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:13.935 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:14.008 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:14.004+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:14.032 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:14.032 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:14.041 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:14.109 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:14.104+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:14.140 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:14.141 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:14.150 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:14.209 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:14.205+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:14.247 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:14.248 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:14.257 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:14.310 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:14.305+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:14.359 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:14.360 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:14.369 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:14.413 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:14.405+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:14.463 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:14.463 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:14.463 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:14.472 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:14.514 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:14.512+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:14.569 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:14.569 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:14.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:14.611 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:14.614 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:14.609+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:14.643 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:14.663 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:14.682 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:14.684 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:14.684 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:14.684 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:14.694 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:14.703 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:14.731 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:14.731 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:14.725+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:14.766 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:14.767 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:14.767 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:14.767 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:14.767 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:14.768 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:14.768 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:14.768 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:14.768 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:14.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:14.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:14.806 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:14.884 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:14.884 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:14.884 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:14.884 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:14.886 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:14.886 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:14.881+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:14.986 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:14.984+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:15.036 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.036 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:15.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.85885576499999, 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:16:15.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.041 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.855184256, 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:15.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:16:15.045 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.853118964, 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.046 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:16:15.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.85310510799999, 2024-09-16T11:16:15.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:16:15.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.054 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.85309123299999, 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:15.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:16:15.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:16:15.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.853077466, 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:16:15.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:16:15.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.85302995699999, 2024-09-16T11:16:15.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:16:15.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:16:15.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.85274413900001, 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.068 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:16:15.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:16:15.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.76008032300001, 2024-09-16T11:16:15.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:16:15.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:16:15.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:16:15.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 150.75993114100001, 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.078 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:16:15.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:16:15.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771529920999996, 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:16:15.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:16:15.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:16:15.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771482961000004, 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.088 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.090 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771464269000006, 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:16:15.092 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:16:15.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.096 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771448070000005, 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:16:15.100 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771433028000004, 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.101 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771417278000001, 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.105 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.107 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.109 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771401987999994, 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.110 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.112 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771386410000005, 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.115 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:16:15.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:16:15.119 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.771371431000006, 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.120 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:16:15.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:16:15.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:15.124 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.275881679000001, 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.128 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:16:15.129 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:16:15.130 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.131 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.131 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:15.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:15.133 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:15.129+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:15.233 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:15.225+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:15.240 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.240 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:15.240 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:15.240 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:15.251 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:15.334 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:15.329+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:15.352 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:15.353 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:15.353 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:15.362 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:15.436 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:15.429+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:15.463 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.966029447, 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.464 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:15.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:15.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:16:15.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:16:15.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.712616649, 2024-09-16T11:16:15.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:16:15.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.712347333, 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:16:15.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:16:15.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.711948823, 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.480 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:15.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.484 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.71166003499999, 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:16:15.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:16:15.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.709960619, 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:16:15.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:16:15.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.709882303, 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:15.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.497 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.709848564, 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.498 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:16:15.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:16:15.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.70980781900001, 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:16:15.502 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.504 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.506 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.706607371, 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:16:15.510 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.706589314, 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:16:15.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.706563325, 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:15.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:16:15.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:16:15.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.705864894, 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:15.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:16:15.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:16:15.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.705852335, 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.524 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:16:15.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.527 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:15.528 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 80.796176582000001, 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:16:15.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.532 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:16:15.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.113751077000003, 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:16:15.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:15.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.541 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.542 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.110886183999995, 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:16:15.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:16:15.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:16:15.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.101840268000004, 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.549 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:15.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.554 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:16:15.555 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 71.044424762999995, 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.556 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:16:15.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:16:15.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:15.560 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 70.998165598, 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:16:15.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:16:15.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:15.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.569 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:15.569 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:16:15.571 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:15.571 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:15.565+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:15.671 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:15.665+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:15.716 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:15.717 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:15.717 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:15.726 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:15.772 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:15.768+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:15.818 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.818 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:15.819 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:15.819 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:15.828 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:15.873 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:15.872+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:15.922 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:15.922 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:15.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.84049216299999, 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.923 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.925 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:15.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840477705, 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:15.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:15.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:15.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840460505, 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:15.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840446985, 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.939 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:15.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:15.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840438443, 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:15.944 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:15.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:15.948 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.949 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840428299, 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.950 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:15.951 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.952 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:15.953 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.954 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840414792, 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.955 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.956 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:15.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:15.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:15.960 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.8404012, 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:15.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:15.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:15.965 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840376918, 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:15.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.968 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.970 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840342757, 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:15.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.84030936000001, 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:15.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:15.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840286715, 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.981 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:15.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.985 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.84027114, 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.986 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:15.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.840254106, 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.991 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:15.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.836621142, 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:15.996 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:15.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:15.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:16.001 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.83658680000001, 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:16.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:16.006 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.836570854, 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:16.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:16.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.83655524300001, 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.012 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:16.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:16.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.016 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.83654015499999, 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.017 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:16.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.020 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.021 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.836525353, 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.022 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:16.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:16.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.027 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:16.029 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:16.029 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:16.024+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:16.129 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:16.124+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:16.136 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:16.136 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:16.136 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:16.146 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:16.231 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:16.225+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:16.242 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:16.242 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:16.242 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:16.242 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:16.242 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:16.252 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:16.333 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:16.325+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:16.398 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.260717632, 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:16.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.26069924399999, 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.404 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.408 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.260398487, 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:16.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.412 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.260368782, 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:16.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:16.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.26035538400001, 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:16.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:16.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.26034157999999, 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:16.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:16.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.25996658099999, 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.426 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:16.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.24642255000001, 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:16.430 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.434 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.24640097599999, 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:16.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:16.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.24638447699999, 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.439 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:16.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:16.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.24636840100001, 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.443 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:16.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:16.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.245984827, 2024-09-16T11:16:16.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:16.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:16.450 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.23389444899999, 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.453 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:16.454 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:16.455 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.23388642699999, 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.456 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:16.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:16.461 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.550407548, 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:16.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:16.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:16.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.466 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:16.467 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 114.316989197, 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:16.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:16.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:16.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.472 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.93059582399999, 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.473 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:16.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:16.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:16.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 102.97899117599999, 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:16.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:16.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 95.843194244000003, 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:16.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 28.790470326000001, 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:16.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:16.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:16.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:16.495 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:16.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:16.497 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:16.489+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:16.499 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:16:16.597 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:16.592+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:16.603 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:16.603 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:16.603 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:16.612 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:16.698 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:16.696+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:16.720 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:16.720 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:16.721 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:16.721 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:16.721 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:16.730 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:16.773 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:16.799 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:16.796+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.801579785, 2024-09-16T11:16:16.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.881 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:16.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.801539023, 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:16.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:16.890 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.80150822500001, 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.891 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:16.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:16.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.801489637, 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.896 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:16.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.899 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.801471585, 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.901 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:16.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.904 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.906 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.801451658, 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.907 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:16.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.801432819, 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.912 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:16.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:16.917 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.80124087, 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:16.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.922 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.80122159, 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.923 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:16.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:16.927 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797730022, 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.928 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:16.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797719951, 2024-09-16T11:16:16.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:16.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797691648, 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:16.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:16.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:16.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:16.943 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.79767771900001, 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.944 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:16.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797668956, 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:16.949 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:16.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.954 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797645137, 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.955 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:16.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.79761401499999, 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.960 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:16.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:16.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.964 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797566276, 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:16.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:16.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 114.797544168, 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.970 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.971 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.973 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:16.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:16.975 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.388349804000001, 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:16.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:16.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:16.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 63.388303823999998, 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:16.983 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:16.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:16.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:16.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.990 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.991 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:16.991 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:16.991 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:16.992 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:16.992 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:16.992 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:16.993 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:16.993 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:16.985+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:16.993 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:17.003 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:17.025 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:17.047 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:17.048 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:17.052 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:17.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:17.093 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:17.088+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:17.099 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.099 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.109 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:17.194 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:17.189+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:17.206 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.206 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.216 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:17.295 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:17.292+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:17.315 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.315 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.324 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:17.396 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:17.393+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:17.469 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.469 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:17.497 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:17.492+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:17.581 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.581 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.591 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:17.598 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:17.593+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:17.693 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.694 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.699 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:17.696+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:17.704 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:17.800 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:17.793+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:17.811 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.812 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.821 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:17.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4] out_osds: [5, 7, 1] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-16T11:16:17.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:16:17.863 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-16T11:16:17.863 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-16T11:16:17.901 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:17.896+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:17.940 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:17.940 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:17.950 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:18.002 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:18.000+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:18.044 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.045 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.055 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:18.104 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:18.100+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:18.147 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.147 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.156 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:18.167 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:16:18.161+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e190 definitely_dead 0 2024-09-16T11:16:18.204 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:18.200+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:18.245 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.245 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.255 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:18.305 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:18.297+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:18.358 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.358 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.367 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:18.406 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:18.401+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:18.467 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.467 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.476 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:18.506 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:18.504+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:18.517 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:16:18.534 INFO:teuthology.orchestra.run.smithi162.stderr:marked in osd.5. 2024-09-16T11:16:18.546 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-16T11:16:18.607 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:18.601+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:18.617 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.617 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.627 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:18.708 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:18.704+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:18.744 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.744 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.754 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:18.809 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:18.808+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:18.849 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.849 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.859 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:18.910 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:18.905+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:18.957 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:18.957 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:18.966 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:19.010 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:19.005+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:19.054 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:19.065 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:19.065 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:19.075 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:19.117 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:19.113+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:19.184 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:19.184 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:19.186 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:19.194 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:19.204 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:19.211 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:19.205+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:19.223 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:19.242 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:19.263 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:19.284 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:19.295 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:19.295 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:19.295 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:19.295 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:19.295 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:19.304 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:19.307 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:19.308 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:19.311 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:19.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:19.312 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:19.308+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:19.398 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.32023840700001, 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:19.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:19.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:19.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:19.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:19.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.405 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.32019764499999, 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.406 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:19.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:19.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:19.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.411 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.320166847, 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:19.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:19.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:19.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.32014825900001, 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:19.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:19.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.422 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.32013020700001, 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:19.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.427 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.32011027999999, 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:19.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:19.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:19.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.320091441, 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.433 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.319899492, 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.438 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:19.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:19.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.319880212, 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:19.448 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.316388644, 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:19.449 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.450 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.451 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:19.452 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.453 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.31637857299999, 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.455 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:19.456 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.457 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:19.458 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.459 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.31635027, 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.460 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:19.461 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:19.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.463 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.464 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.316336341, 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:19.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.466 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.467 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.468 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:19.469 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.316327578, 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:19.470 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.471 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.472 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:19.473 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:19.475 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.31630375899999, 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:19.476 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.477 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:19.478 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.479 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.316272637, 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:19.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:19.482 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:19.483 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:19.484 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.485 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.316224898, 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.486 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.487 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:19.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:19.489 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.490 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.31620279000001, 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:19.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.492 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.493 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:19.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 65.907008426000004, 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.496 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:19.497 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.498 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.499 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:19.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.501 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.502 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 65.906962445999994, 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:19.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.504 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.505 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.506 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:19.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:19.508 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.509 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.510 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:19.512 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:19.512 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:19.505+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:19.540 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:16:19.610 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:19.610 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:19.610 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:19.610 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:19.612 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:19.608+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:19.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:19.714 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:19.709+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:19.716 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:19.716 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:19.717 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:19.717 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:19.726 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:19.815 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:19.812+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:19.823 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 178.95795868499999, 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:16:19.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:16:19.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78998703600001, 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:16:19.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.833 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78997547500001, 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.834 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:16:19.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78996227299999, 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:16:19.838 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:16:19.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:16:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78994983300001, 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.843 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:19.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:16:19.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78993632199999, 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:19.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:16:19.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.789925407, 2024-09-16T11:16:19.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78989948899999, 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:16:19.857 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:16:19.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.789884035, 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:16:19.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:16:19.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78987414900001, 2024-09-16T11:16:19.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:16:19.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.789864386, 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.871 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:19.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:16:19.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.789854209, 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.876 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:16:19.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:16:19.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78983772699999, 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.881 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.882 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:16:19.883 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.884 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78982303000001, 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:16:19.885 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.886 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:16:19.887 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:16:19.888 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.889 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78980458699999, 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.890 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:19.891 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.892 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.893 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78978763800001, 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:16:19.894 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.895 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.896 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.897 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:16:19.898 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78976842500001, 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.899 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.900 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.901 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.902 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:16:19.903 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.789751209, 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.904 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.905 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:16:19.906 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.907 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78972995500001, 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.910 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.911 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 151.78970998700001, 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:19.913 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:19.914 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:16:19.916 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:19.917 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:19.918 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:19.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:19.920 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:19.917+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:20.020 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:20.013+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:20.069 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:20.069 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:20.069 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:20.069 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:20.079 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:20.121 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:20.115+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:20.180 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:20.180 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:20.180 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:20.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:20.222 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:20.217+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:20.292 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:20.292 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:20.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:20.292 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:20.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.292 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.09040905200001, 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.293 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:20.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.09039298499999, 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.297 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.301 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.09037659000001, 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:20.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:20.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:20.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.09036701599999, 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:20.306 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:20.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 152.09035747999999, 2024-09-16T11:16:20.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:20.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:20.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.392520597, 2024-09-16T11:16:20.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:20.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 108.706685343, 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.319 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:20.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:20.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:20.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:20.324 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:20.325 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.772165999999999, 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:20.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.330 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.772085329000006, 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:20.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:20.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.335 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.771963936999995, 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:20.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.340 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.771911743000004, 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.341 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:20.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.345 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.771855009999996, 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:20.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:20.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.771807811000002, 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:20.351 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:20.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:20.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.356 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.771757840999996, 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.357 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:20.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.771690727000006, 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.362 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:20.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:20.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.769715641000005, 2024-09-16T11:16:20.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:20.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:20.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:20.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.769667155999997, 2024-09-16T11:16:20.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:20.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:20.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.377 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.769550128000006, 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.378 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:20.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:20.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 74.769458952999997, 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:20.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:20.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.387 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 47.529052106000002, 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.388 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:20.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:20.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:20.394 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:20.396 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:20.391+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:20.396 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:20.497 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:20.489+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:20.509 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:20.510 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:20.510 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:20.510 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:20.519 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:20.598 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:20.593+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:20.615 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:20.616 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:20.616 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:20.616 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:20.616 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:20.625 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:20.699 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:20.693+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:20.726 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:20.726 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.54751766499999, 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.727 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:20.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:16:20.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.731 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.543846156, 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.732 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:16:20.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.541780864, 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:16:20.737 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.738 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:16:20.739 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:16:20.740 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.54176700799999, 2024-09-16T11:16:20.741 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.742 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.744 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.745 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.54175313299999, 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.746 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:20.747 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.748 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.749 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:16:20.750 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.541739366, 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.751 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.752 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:16:20.753 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:16:20.754 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:20.755 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.54169185699999, 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:20.756 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.757 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:16:20.758 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:16:20.759 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.54140603900001, 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.760 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.761 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:16:20.762 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.763 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:16:20.764 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.44874222300001, 2024-09-16T11:16:20.765 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.766 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.767 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.768 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.769 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:20.770 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:20.771 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.44859304100001, 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.772 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:20.773 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.774 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:16:20.776 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.777 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:16:20.778 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460191820999995, 2024-09-16T11:16:20.779 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:20.780 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.781 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.782 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:16:20.783 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460144861000003, 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.784 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.785 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.786 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.787 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.788 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460126169000006, 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.789 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:20.790 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.791 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.792 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460109970000005, 2024-09-16T11:16:20.793 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.794 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.795 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.796 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:16:20.797 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460094928000004, 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.798 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.799 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:20.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.801 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.802 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460079178000001, 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.803 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:16:20.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.805 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.806 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.807 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460063887999993, 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.808 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.809 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:16:20.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:16:20.811 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460048310000005, 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.812 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.813 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.814 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.815 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:16:20.816 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.460033331000005, 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.817 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.819 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:16:20.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.821 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 47.964543579000001, 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:20.822 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:20.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.824 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:16:20.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:16:20.826 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:16:20.827 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.828 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:20.829 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:20.829 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:20.830 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:20.832 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:20.827+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:20.931 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:20.925+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:20.950 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:20.951 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:20.951 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:20.960 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:21.032 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:21.025+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:21.104 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:21.104 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:21.104 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:21.104 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:21.104 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:21.112 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:21.158 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:21.153+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:21.220 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:21.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.722268273, 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:16:21.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:16:21.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:16:21.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.468855475, 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.468586159, 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:16:21.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:16:21.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:21.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.468187649, 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:16:21.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.46789886099999, 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.242 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.246 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.466199445, 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:16:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.250 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.466121129, 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:16:21.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:16:21.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.46608739, 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:16:21.255 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.256 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:16:21.257 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.258 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.259 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.46604664500001, 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:16:21.260 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:16:21.261 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.262 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.263 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.462846197, 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.264 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.265 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.266 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.267 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.268 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.46282814, 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:21.269 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.270 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.271 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.272 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.46280215100001, 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.273 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.274 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.275 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.276 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.46210372, 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:16:21.277 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.278 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.279 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:16:21.280 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.281 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.462091161, 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.282 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.283 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.284 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:16:21.285 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 86.552415408000002, 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.286 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.287 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.288 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:16:21.290 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:21.291 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.292 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:21.293 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.869989903000004, 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:16:21.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.867125009999995, 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.300 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.302 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:16:21.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.858079094000004, 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:16:21.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:16:21.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:16:21.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.313 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.800663588999996, 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:16:21.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:16:21.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 76.754404424000001, 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:16:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:21.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:21.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:21.327 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:21.328 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:21.329 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:21.323+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:21.329 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:21.430 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:21.425+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:21.445 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:21.445 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:21.445 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:21.460 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:21.471 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:21.489 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:21.507 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:21.527 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:21.530 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:21.525+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:21.544 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:16:21.549 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:21.566 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:21.566 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:21.575 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:21.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:21.596 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:21.597 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:21.599 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:21.599 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:21.645 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:21.641+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:21.717 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:21.717 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:21.727 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:21.731 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:21.725+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:21.826 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:21.826 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:21.832 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:21.825+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:21.835 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:21.933 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:21.925+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:21.988 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:21.989 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:21.998 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:22.034 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:22.031+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:22.103 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.103 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.113 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:22.135 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:22.129+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:22.222 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.223 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.233 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:22.236 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:22.231+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:22.336 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:22.329+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:22.342 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.342 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.352 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:22.437 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:22.433+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:22.469 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.469 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:22.538 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:22.533+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:22.566 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:16:22.587 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.588 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.599 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:22.640 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:22.633+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:22.707 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.708 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.718 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:22.740 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:22.733+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:22.809 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.809 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.821 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:22.870 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:22.867+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:22.916 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:22.916 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:22.926 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:22.942 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:22.939+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:23.023 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:23.023 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:23.033 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:23.044 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:23.037+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:23.145 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:23.139+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:23.189 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:23.189 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:23.199 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:23.246 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:23.243+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:23.306 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:23.306 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:23.316 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:23.347 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:23.343+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:23.419 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:23.419 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:23.428 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:23.448 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:23.441+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:23.535 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:23.535 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:23.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:23.547 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4, 5] out_osds: [7, 1] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-16T11:16:23.547 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:16:23.547 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 3, 6, 2, 4, 5] 2024-09-16T11:16:23.548 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-16T11:16:23.549 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:23.543+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:23.600 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:23.650 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:23.645+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:23.688 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:23.689 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:23.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:23.741 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:23.751 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:23.747+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:23.760 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:23.777 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:23.794 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:23.813 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:23.834 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:23.842 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:23.843 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:23.843 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:23.843 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:23.843 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:23.852 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:23.853 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:23.847+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:23.856 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:23.856 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:23.857 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:23.859 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:23.860 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:23.908 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:16:23.901+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e196 definitely_dead 0 2024-09-16T11:16:23.954 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:23.949+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:23.955 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 136.458481049, 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.956 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.957 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:16:23.958 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:23.959 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:16:23.960 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:23.961 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.962 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:23.963 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.205068251, 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.964 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:16:23.965 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:16:23.966 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.967 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.204798935, 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.968 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:16:23.969 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:16:23.970 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.971 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.204400425, 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.972 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.973 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:16:23.974 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:16:23.975 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.204111637, 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:16:23.976 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.977 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.978 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.979 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.980 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.202412221, 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:23.981 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:16:23.982 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.983 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.984 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.202333905, 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.985 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:16:23.986 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.987 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.988 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.202300166, 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.989 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.990 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.991 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:16:23.992 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.993 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.20225942099999, 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.994 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.995 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.996 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:16:23.997 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.19905897300001, 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:23.998 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:23.999 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:16:24.000 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.001 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.199040916, 2024-09-16T11:16:24.002 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.003 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.004 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.005 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.006 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.19901492699999, 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.007 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:24.008 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:16:24.009 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.010 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:24.011 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.198316496, 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.012 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:16:24.013 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.014 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.015 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.19830393700001, 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.016 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:16:24.017 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:16:24.018 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.019 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 89.288628184000004, 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.020 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:24.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:24.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.027 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 79.606202679000006, 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:16:24.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:16:24.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:16:24.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.034 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 79.603337785999997, 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:16:24.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.037 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:16:24.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.041 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 79.594291870000006, 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.042 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:16:24.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:16:24.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:24.051 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 79.536876364999998, 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:24.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.053 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:16:24.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 79.490617200000003, 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.058 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:24.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.064 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.066 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:24.066 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:24.061+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:24.167 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:24.163+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:24.180 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:24.180 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:24.180 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:24.180 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.189 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:24.268 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:24.263+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:24.285 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:24.285 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:24.285 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:24.285 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:24.285 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.295 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:24.369 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:24.365+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:24.404 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:24.404 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:24.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:24.404 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:24.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.404 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.322093245, 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:24.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:24.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.408 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:24.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.322078787, 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:24.410 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.416 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.32206158699999, 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:24.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:24.418 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.419 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:24.420 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:24.421 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.422 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.322048067, 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.423 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:24.424 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.425 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.426 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.427 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.32203952499999, 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.428 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:24.429 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.430 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:24.431 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.432 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.433 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.32202938099999, 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.434 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.435 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:24.436 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.437 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:24.438 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.322015874, 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:24.439 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.440 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.441 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:24.442 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.443 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.444 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.322002282, 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.445 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.447 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.448 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:24.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.450 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.450 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.450 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:24.450 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.321978, 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.451 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:24.452 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.453 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.456 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.457 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.458 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.459 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.321943839, 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.460 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.461 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.462 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:24.463 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.464 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.465 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.321910442, 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.466 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.467 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.468 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:24.469 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.470 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:24.471 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.321887797, 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.472 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:24.473 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.474 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:24.475 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.476 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.477 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.321872222, 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.478 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.479 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:24.480 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:24.481 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.321855188, 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.483 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:24.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:24.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:24.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.318222224, 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:24.488 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:24.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:24.493 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.318187882, 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:24.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.498 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.318171936, 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:24.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:24.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.503 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.318156325, 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:24.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:24.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:24.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.31814123700001, 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:24.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.318126435, 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:24.514 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:24.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:24.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:24.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.520 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.520 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.520 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.522 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:24.522 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:24.515+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:24.574 INFO:teuthology.orchestra.run.smithi162.stderr:marked out osd.5. 2024-09-16T11:16:24.580 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T11:16:24.622 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:24.619+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:24.636 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:24.636 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.646 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:24.723 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:24.719+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:24.743 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:24.744 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:24.744 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.753 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:24.824 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:24.819+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:24.849 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.712434522, 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.850 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:24.851 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.852 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:24.853 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.854 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.71241613399999, 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.855 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:24.856 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.857 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.858 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.712115377, 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.859 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.860 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:24.861 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.862 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:24.863 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.712085672, 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:24.864 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.865 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.866 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.867 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.71207227400001, 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.868 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.869 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.870 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.871 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.872 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.71205846999999, 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.873 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.874 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:24.875 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.876 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.71168347099999, 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.877 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:24.878 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.879 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.880 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.881 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.69813944000001, 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.882 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.883 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:24.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.885 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:24.886 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.69811786599999, 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.887 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:24.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:24.889 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.890 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.69810136699999, 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.891 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.892 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:24.893 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:24.894 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.69808529100001, 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.895 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:24.896 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.897 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:24.898 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.899 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.697701717, 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.900 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:24.901 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.902 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.903 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.68561133899999, 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:24.904 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.905 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.906 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:24.907 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.908 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 156.68560331699999, 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.909 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:24.910 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.911 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.912 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.00212443800001, 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.913 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.914 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.915 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.916 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:24.917 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.918 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 122.768706087, 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.919 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.920 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:24.921 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:24.922 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.923 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:24.924 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 113.38231271399999, 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.925 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.926 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.927 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.928 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.929 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 111.43070806599999, 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:24.930 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:24.931 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.932 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.933 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.934 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:24.935 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 104.294911134, 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.936 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.937 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:24.938 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:24.939 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.940 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:24.941 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 37.242187215999998, 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.942 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:24.943 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:24.944 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:24.945 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.946 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:24.947 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:24.948 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:24.949 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:24.950 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:24.943+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:25.050 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:25.045+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:25.057 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.058 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:25.058 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:25.058 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.067 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:25.152 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:25.147+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:25.165 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.166 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:25.166 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:25.166 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:25.166 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.175 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:25.253 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:25.249+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:25.274 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.274 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:25.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:25.274 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.19577859899999, 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:25.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.281 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.195737837, 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.282 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.284 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.195707039, 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.288 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:25.289 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.290 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:25.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.292 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.195688451, 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.293 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:25.294 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:25.296 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.297 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.298 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.19567039899999, 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.195650472, 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.195631633, 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.309 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:25.310 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:25.311 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.312 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.313 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.19543968399999, 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:25.314 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:25.315 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.316 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.317 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.318 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.195420404, 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:25.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.320 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.321 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.322 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:25.323 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.191928836, 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.324 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.327 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.328 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.191918765, 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:25.329 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.330 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:25.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:25.332 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.333 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.334 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.191890462, 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.335 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.337 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.338 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.339 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.191876533, 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.340 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.341 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.342 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:25.343 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.344 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.19186777, 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.345 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:25.346 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.347 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.348 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:25.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.350 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.191843951, 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:25.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.191812829, 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.356 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:25.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:25.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.19176509, 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.361 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:25.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.19174298199999, 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.366 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.367 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:25.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 71.782548618000007, 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.371 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.372 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:25.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 71.782502637999997, 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.378 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.379 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:25.380 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:25.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:25.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.385 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:25.387 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:25.381+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:25.475 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is set 2024-09-16T11:16:25.487 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:25.481+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:25.513 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.513 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:25.513 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:25.513 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.523 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:25.589 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:25.581+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:25.618 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.618 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:25.618 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:25.619 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:25.619 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.628 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:25.690 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:25.683+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 184.85908702, 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.726 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:16:25.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.730 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.691115371, 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:16:25.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:16:25.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:16:25.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69110380999999, 2024-09-16T11:16:25.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.739 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.691090608, 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.740 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:25.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69107816799999, 2024-09-16T11:16:25.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:16:25.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.748 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.691064657, 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.749 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69105374200001, 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:16:25.753 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:16:25.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:16:25.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.691027824, 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:16:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.762 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69101237000001, 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:25.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.766 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69100248399999, 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69099272099999, 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.775 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69098254400001, 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.779 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.690966062, 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:16:25.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69095136499999, 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:16:25.784 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:16:25.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.690932922, 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.789 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:16:25.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69091597299999, 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.793 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.797 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69089675999999, 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:25.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69087954400001, 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.802 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:16:25.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69085828999999, 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:16:25.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 157.69083832199999, 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:25.811 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:16:25.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:16:25.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:25.815 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.817 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:25.817 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:25.809+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:25.861 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:25.918 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:25.913+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:25.963 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:25.964 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:25.964 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:25.964 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:25.973 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:26.009 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:26.018 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:26.015+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:26.030 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:26.052 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:26.069 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:26.069 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:26.070 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:26.070 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:26.070 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:26.074 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:26.079 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:26.096 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:26.117 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:26.119 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:26.113+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:26.138 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:26.139 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:26.139 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:26.139 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:26.139 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:26.140 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:26.140 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:26.140 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:26.140 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:26.143 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:26.144 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:26.221 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:26.213+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:26.221 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.221 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.230 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:26.322 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:26.317+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:26.339 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.339 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.349 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:26.423 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:26.419+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:26.451 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.451 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.460 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:26.523 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:26.517+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:26.565 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.565 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:26.625 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:26.619+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:26.683 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.683 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.693 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:26.726 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:26.723+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:26.807 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.808 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.819 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:26.827 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:26.823+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:26.916 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:26.916 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:26.924 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:26.927 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:26.923+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:27.023 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.023 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.028 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:27.023+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:27.032 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:27.128 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.128 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.129 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:27.125+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:27.137 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:27.230 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:27.225+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:27.237 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.237 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.246 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:27.331 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:27.327+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:27.395 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.395 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.405 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:27.433 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:27.425+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:27.489 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T11:16:27.505 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.505 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.514 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:27.534 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:27.527+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:27.634 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:27.629+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:27.670 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.670 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.681 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:27.737 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:27.729+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:27.788 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.788 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.797 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:27.838 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:27.835+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:27.899 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:27.899 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:27.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:27.941 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:27.935+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:28.008 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:28.008 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:28.017 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:28.058 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:28.055+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:28.143 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:28.139+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:28.145 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:28.157 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:28.157 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:28.166 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:28.244 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:28.236+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:28.266 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:28.266 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:28.275 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:28.294 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:28.315 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:28.336 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:28.344 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:28.339+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:28.356 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:28.377 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:28.377 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:28.385 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:28.386 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:28.404 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:28.423 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:28.424 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:28.424 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:28.424 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:28.425 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:28.425 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:28.425 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:28.425 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:28.425 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:28.428 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:28.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:28.445 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:28.439+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:28.489 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:28.489 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:28.489 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:28.498 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:28.546 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:28.540+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:28.598 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is set 2024-09-16T11:16:28.601 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:28.601 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:28.602 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:28.602 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:28.602 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:28.612 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:28.647 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:28.640+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:28.706 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:28.706 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:28.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:28.706 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 160.50590583100001, 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:28.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:28.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:28.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:28.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:28.712 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 160.50588976399999, 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:28.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.716 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 160.505873369, 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:28.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:28.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:28.721 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 160.50586379500001, 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:28.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 160.50585425899999, 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.726 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:28.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.729 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.80801737600001, 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.731 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:28.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:28.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:28.734 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 117.122182122, 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:28.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.736 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:28.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:28.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:28.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:28.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187662778999993, 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:28.741 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:28.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.746 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187582108000001, 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:28.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:28.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.751 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187460716000004, 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.752 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:28.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.757 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187408521999998, 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:28.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:28.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187351789000004, 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:28.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.765 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:28.767 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187304589999997, 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.768 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:28.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187254620000004, 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.773 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:28.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:28.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.187187506000001, 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:28.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:28.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.784 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.185212419999999, 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.785 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:28.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.185163935000006, 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:28.790 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:28.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.185046907, 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.796 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:28.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:28.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.801 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.184955732000006, 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.802 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:28.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.806 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 55.944548885000003, 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.807 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:28.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:28.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:28.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:28.813 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:28.815 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:28.815 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:28.808+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:28.916 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:28.908+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:28.925 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:28.925 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:28.925 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:28.925 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:28.934 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:29.017 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:29.011+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:29.039 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.040 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:29.040 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:29.040 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:29.040 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.049 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:29.118 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:29.112+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42142 4.16 4:6caa20e2:::rbd_data.11b5f978907f.0000000000000007:f [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:29.154 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.975038084, 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503262000000005, 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42142 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:29.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174277+0000", 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174278+0000", 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2399999999999999e-07 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174280+0000", 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0760000000000001e-06 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174285+0000", 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1780000000000002e-06 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.174288+0000", 2024-09-16T11:16:29.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4199999999999999e-06 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.236587+0000", 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.062298113000000002 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237044+0000", 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00045782899999999998 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.237781+0000", 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00073632199999999997 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.97136657499999, 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.96930128299999, 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:16:29.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.96928742700001, 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:29.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:16:29.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.969273552, 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:29.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.177 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.179 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.96925978499999, 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:29.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.96921227600001, 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.184 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:16:29.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.968926458, 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:16:29.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:16:29.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:16:29.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:16:29.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.876262642, 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:16:29.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:16:29.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:16:29.199 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 164.87611346, 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.200 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:29.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:16:29.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887712239999999, 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:16:29.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887665279999993, 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:16:29.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887646587999996, 2024-09-16T11:16:29.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:16:29.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887630388999995, 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:16:29.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:16:29.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887615346999993, 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:16:29.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:29.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:16:29.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887599597000005, 2024-09-16T11:16:29.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:16:29.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.233 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887584306999997, 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.234 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:16:29.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:16:29.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887568728999995, 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.239 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:29.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:16:29.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 84.887553749999995, 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.244 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:16:29.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:16:29.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:29.248 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 56.392063997999998, 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.249 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.250 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.251 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:16:29.252 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:29.253 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.254 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.255 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:29.258 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:29.248+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:29.357 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:29.351+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:29.406 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:29.406 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:29.406 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.415 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:29.458 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:29.455+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:29.513 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.513 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:29.513 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:29.513 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:29.513 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.522 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:29.559 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:29.552+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:29.587 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 6, 2, 4] out_osds: [7, 1, 5] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-16T11:16:29.588 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T11:16:29.588 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-16T11:16:29.588 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-16T11:16:29.632 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.632 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:29.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:29.632 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:29.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 142.13606301999999, 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:16:29.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:16:29.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:16:29.639 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.882650222, 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.640 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.88238090599999, 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.644 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.648 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.881982396, 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.88169360800001, 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.653 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:16:29.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:16:29.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.879994192, 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.657 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:16:29.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:16:29.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.879915876, 2024-09-16T11:16:29.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:16:29.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.879882137, 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:16:29.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:16:29.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.879841392, 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.670 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:16:29.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:16:29.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.876640944, 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:16:29.674 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:16:29.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:16:29.678 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.876622887, 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:16:29.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.876596898, 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.683 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.684 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:16:29.685 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.686 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.687 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.875898467, 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:29.688 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.689 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.690 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.691 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 127.875885908, 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.692 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.693 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.694 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.695 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 94.966210154999999, 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.696 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.697 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.698 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.699 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:16:29.700 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:16:29.701 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.702 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:29.703 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.283784650000001, 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.704 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:16:29.705 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:16:29.706 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.707 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.708 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:16:29.709 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.280919757000007, 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.710 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.711 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.712 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.713 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.714 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:16:29.715 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.271873841000001, 2024-09-16T11:16:29.716 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.717 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:16:29.718 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:16:29.719 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:16:29.720 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.721 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.722 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.214458336000007, 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.723 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:16:29.724 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.725 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:16:29.726 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:29.727 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.728 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:29.729 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 85.168199170999998, 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.730 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.731 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.732 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:16:29.733 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:16:29.734 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:29.735 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:29.736 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.737 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:29.738 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:29.728+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:29.838 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:29.832+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:29.881 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.881 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:29.881 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:29.881 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.889 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:29.904 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:16:29.896+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e202 definitely_dead 0 2024-09-16T11:16:29.938 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:29.932+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:29.983 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:29.984 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:29.984 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:29.984 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:29.984 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:29.993 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:30.039 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:30.032+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:30.138 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:30.138 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:30.138 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:30.138 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05644240699999, 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.139 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.140 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:30.141 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:30.142 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.143 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05642794900001, 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.144 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.145 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:30.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:30.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:30.149 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05641074900001, 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:30.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:30.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:30.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:30.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.056397229, 2024-09-16T11:16:30.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:30.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:30.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.159 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05638868700001, 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:30.161 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:30.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:30.166 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05637854299999, 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.168 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.171 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05636503599999, 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:30.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:30.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:30.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.056351444, 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:30.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:30.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.056327162, 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.182 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:30.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.056293001, 2024-09-16T11:16:30.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:30.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.192 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05625960399999, 2024-09-16T11:16:30.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:30.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:30.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.200 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05623695899999, 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.201 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:30.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:30.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.056221384, 2024-09-16T11:16:30.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:30.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:30.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:30.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:30.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05620435, 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:30.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05257138600001, 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:30.221 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05253704399999, 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.222 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:30.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:30.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.052521098, 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.227 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:30.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:30.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05250548699999, 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:30.232 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:30.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:30.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:30.237 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05249039899999, 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:30.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.240 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.241 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.05247559700001, 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.243 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.244 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:30.245 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:30.246 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.247 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.248 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.248 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:30.249 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:30.250 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:30.243+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:30.350 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:30.344+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:30.357 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:30.357 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:30.357 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:30.357 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:30.365 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:30.430 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T11:16:30.451 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:30.447+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:30.475 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:30.476 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:30.476 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:30.476 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:30.476 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:30.482 INFO:teuthology.orchestra.run.smithi162.stderr:marked in osd.1. 2024-09-16T11:16:30.490 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:30.493 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-16T11:16:30.553 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:30.547+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:30.572 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.44366144899999, 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:30.580 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:30.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.584 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.44364306099999, 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.585 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:30.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.589 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.443342304, 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:30.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.443312599, 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:30.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.443299201, 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:30.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:30.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.44328539700001, 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:30.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.44291039800001, 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.611 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:30.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.429366367, 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.616 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:30.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.42934479300001, 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:30.620 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:30.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.42932829399999, 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:30.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:30.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.42931221800001, 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.629 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.633 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.428928644, 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.634 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:30.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.41683826600001, 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:30.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:30.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 162.41683024400001, 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:30.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.647 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.733351365, 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:30.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:30.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:30.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 128.49993301399999, 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:30.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.113539641, 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:30.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 117.161934993, 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:30.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:30.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:30.670 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 110.026138061, 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.671 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:30.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 42.973414142999999, 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:30.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:30.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:30.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.680 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.681 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:30.682 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:30.683 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:30.683 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:16:30.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:30.686 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:30.686 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:30.686 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:30.686 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:30.686 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:30.687 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:30.679+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:30.706 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:30.706 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:30.707 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:30.710 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:30.711 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:30.786 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:30.779+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:30.791 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:30.791 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:30.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:30.886 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:30.883+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:30.899 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:30.899 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:30.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:30.987 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:30.983+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:31.055 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.055 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:31.088 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:31.080+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:31.167 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.167 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.176 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:31.189 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:31.180+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:31.274 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.274 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.282 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:31.289 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:31.284+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:31.378 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.378 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:31.389 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:31.384+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:31.484 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:16:31.490 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:31.487+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:31.490 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.490 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.499 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:31.591 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:31.584+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:31.646 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.647 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.655 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:31.692 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:31.684+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:31.750 INFO:teuthology.orchestra.run.smithi165.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.750 INFO:teuthology.orchestra.run.smithi165.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.759 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:31.793 INFO:tasks.ceph.osd.4.smithi165.stderr:2024-09-16T11:16:31.787+0000 7f08ae5c0700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22369) UID: 0 2024-09-16T11:16:31.871 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.871 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.880 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:31.893 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:31.887+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:31.980 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:31.980 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:31.989 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:31.993 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:31.988+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:32.085 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.085 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.094 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:32.094 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:32.088+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:32.195 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:32.191+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:32.240 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.240 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.250 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:32.296 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:32.291+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:32.351 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.352 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.360 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:32.406 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:32.400+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:32.462 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.463 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.479 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:32.498 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:32.495+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:32.599 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:32.595+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:32.609 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.609 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.620 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:32.699 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:32.692+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:32.712 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T11:16:32.735 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.735 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.747 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:32.800 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:32.792+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:32.855 INFO:teuthology.orchestra.run.smithi162.stdout:osd.0: {} 2024-09-16T11:16:32.885 INFO:teuthology.orchestra.run.smithi162.stdout:osd.1: {} 2024-09-16T11:16:32.901 INFO:teuthology.orchestra.run.smithi162.stdout:ERROR: (22) Invalid argument 2024-09-16T11:16:32.901 INFO:teuthology.orchestra.run.smithi162.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T11:16:32.901 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:32.895+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:32.904 INFO:teuthology.orchestra.run.smithi162.stdout:osd.2: {} 2024-09-16T11:16:32.911 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:32.923 INFO:teuthology.orchestra.run.smithi162.stdout:osd.3: {} 2024-09-16T11:16:32.948 INFO:teuthology.orchestra.run.smithi162.stdout:osd.4: {} 2024-09-16T11:16:32.983 INFO:teuthology.orchestra.run.smithi162.stdout:osd.5: {} 2024-09-16T11:16:33.002 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:32.996+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:33.016 INFO:teuthology.orchestra.run.smithi162.stdout:osd.6: {} 2024-09-16T11:16:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T11:16:33.016 INFO:teuthology.orchestra.run.smithi162.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T11:16:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T11:16:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T11:16:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T11:16:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T11:16:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T11:16:33.017 INFO:teuthology.orchestra.run.smithi162.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T11:16:33.020 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T11:16:33.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T11:16:33.025 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:33.025 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:33.025 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:33.025 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:33.035 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:33.103 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:33.096+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:33.177 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:33.178 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:33.178 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:33.187 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:33.204 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:33.196+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:33.291 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.152922442, 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.292 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.293 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:33.294 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.295 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:33.296 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.152904054, 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.297 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.298 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.299 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.300 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.301 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.15260329700001, 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.302 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:33.303 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:33.304 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.305 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.15257359200001, 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.306 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.307 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:33.308 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.309 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:33.310 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.15256019399999, 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.311 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:33.312 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.313 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.314 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.15254639, 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.315 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:33.316 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:33.317 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:33.318 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.152171391, 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.319 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.320 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:33.321 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.322 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:33.323 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.13862735999999, 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.324 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:33.325 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.326 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.327 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.138605786, 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.328 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:33.329 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:33.330 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:33.331 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.138589287, 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.332 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.333 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.334 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:33.335 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.13857321099999, 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:33.336 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.337 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:33.338 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.339 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.340 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.13818963700001, 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.341 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.342 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.343 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:33.344 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.345 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.346 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:33.347 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:33.348 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.349 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.126099259, 2024-09-16T11:16:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:33.350 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:33.351 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.352 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.353 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.354 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.355 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.356 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:33.357 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:33.358 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:33.359 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.360 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 165.126091237, 2024-09-16T11:16:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.361 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.362 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:33.363 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:33.364 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.365 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:33.366 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.367 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 136.44261235799999, 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.368 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:33.369 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.370 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.371 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.372 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.373 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:33.374 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 131.20919400700001, 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.375 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.376 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.377 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.378 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:33.379 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.380 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.381 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.382 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 121.822800634, 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.383 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.384 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.385 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:33.386 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.387 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:33.388 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.389 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 119.871195986, 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:33.390 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:33.391 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:33.392 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:33.393 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:33.394 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.395 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.396 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.397 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 112.735399054, 2024-09-16T11:16:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:33.398 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:33.399 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.400 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.401 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:33.402 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:33.403 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.404 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.405 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.406 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:33.407 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 45.682675136, 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:33.408 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.409 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.410 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.411 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:33.412 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:33.413 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:33.414 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.415 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:33.416 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:33.417 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:33.419 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:33.419 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:33.412+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:33.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:16:33.519 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:33.512+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:33.528 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:33.528 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:33.529 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:33.529 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:33.538 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:33.620 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:33.612+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:33.639 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:33.640 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:33.640 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:33.650 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:33.721 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:33.715+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:33.753 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67684958300001, 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.754 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:33.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:33.759 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67680882100001, 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:33.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:33.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:33.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:33.764 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.676778023, 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.765 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.766 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:33.767 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.768 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.769 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67675943500001, 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:33.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:33.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67674138300001, 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:33.775 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:33.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:33.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:33.780 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.676721456, 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:33.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:33.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67670261699999, 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.786 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:33.787 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.790 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67651066799999, 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:33.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:33.796 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67649138799999, 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:33.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67299982, 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:33.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:33.805 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67298974900001, 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:33.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:33.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.811 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67296144599999, 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:33.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:33.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:33.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67294751700001, 2024-09-16T11:16:33.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:33.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:33.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.672938754, 2024-09-16T11:16:33.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:33.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.827 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67291493499999, 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:33.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.830 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:33.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.672883813, 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:33.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:33.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.672836074, 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:33.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.841 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.842 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 131.67281396600001, 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:33.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:33.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.263619602000006, 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:33.847 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:33.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:33.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 80.263573621999996, 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:33.855 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:33.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:33.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:33.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:33.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:33.862 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:33.864 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:33.864 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:33.856+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:33.964 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:33.956+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:33.969 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:33.970 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:33.970 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:33.979 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:34.066 INFO:tasks.ceph.osd.6.smithi165.stderr:2024-09-16T11:16:34.059+0000 7fad9289a700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22374) UID: 0 2024-09-16T11:16:34.080 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:34.080 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:34.081 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:34.090 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:34.167 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:34.160+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:34.191 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 193.32502255399999, 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:16:34.193 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.194 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.195 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.196 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15705090500001, 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:16:34.197 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.198 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:16:34.199 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:16:34.200 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.201 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.157039344, 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:16:34.202 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:34.203 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.204 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.205 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.206 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15702614200001, 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.207 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.208 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.209 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:16:34.210 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.157013702, 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.211 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:16:34.213 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.214 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:34.215 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15700019100001, 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.217 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.218 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.219 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15698927599999, 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:16:34.220 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.221 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.222 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:16:34.223 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15696335800001, 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:16:34.224 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.225 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.226 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:16:34.227 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15694790399999, 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.229 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:34.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:16:34.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:16:34.232 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15693801800001, 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.233 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.234 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:16:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:16:34.236 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.237 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.156928255, 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.238 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.239 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:16:34.240 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.241 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15691807799999, 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.243 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.244 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.245 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.246 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15690159600001, 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.247 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.248 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:16:34.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:16:34.250 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.251 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.156886899, 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.252 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:16:34.253 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.254 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:16:34.255 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15686845600001, 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:16:34.256 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.257 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.258 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:16:34.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:16:34.260 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.156851507, 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.261 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.262 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.263 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:16:34.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.156832294, 2024-09-16T11:16:34.265 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:16:34.267 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.268 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:16:34.269 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.15681507799999, 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.270 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:16:34.272 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.273 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:34.274 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.156793824, 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:34.275 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.276 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.277 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.278 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.156773856, 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.279 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.280 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.281 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.282 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.283 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.284 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.284 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:34.285 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:34.286 INFO:tasks.ceph.osd.5.smithi165.stderr:2024-09-16T11:16:34.283+0000 7fe51ce48700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22372) UID: 0 2024-09-16T11:16:34.386 INFO:tasks.ceph.osd.0.smithi162.stderr:2024-09-16T11:16:34.380+0000 7f0944fd8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22235) UID: 0 2024-09-16T11:16:34.428 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:34.428 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:34.437 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:34.487 INFO:tasks.ceph.osd.2.smithi162.stderr:2024-09-16T11:16:34.480+0000 7f237a866700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22241) UID: 0 2024-09-16T11:16:34.539 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:34.539 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:34.539 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:34.548 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:34.587 INFO:tasks.ceph.osd.3.smithi162.stderr:2024-09-16T11:16:34.580+0000 7f1675f73700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22239) UID: 0 2024-09-16T11:16:34.642 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:16:34.654 INFO:teuthology.orchestra.run.smithi162.stdout:All operations completed A-OK! 2024-09-16T11:16:34.656 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:34.656 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.45592894800001, 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.657 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.658 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.659 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.660 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:34.661 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.45591288099999, 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:34.662 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.663 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.664 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:34.665 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.455896486, 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.667 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.668 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:34.669 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.45588691200001, 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:34.670 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.671 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:34.672 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:34.673 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 166.45587737599999, 2024-09-16T11:16:34.674 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.675 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:34.676 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.677 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:34.678 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 137.75804049300001, 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.679 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:34.680 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.681 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.682 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 123.072205239, 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.683 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:34.684 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:34.688 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.688 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.688 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.688 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:34.689 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:34.690 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:34.691 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.692 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137685895999994, 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.693 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.694 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.694 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.694 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.694 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:34.694 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.695 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:34.696 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:34.697 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137605225000001, 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:34.703 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.704 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137483833000005, 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.705 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.707 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.708 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:34.709 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137431638999999, 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:34.710 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.711 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.712 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:34.713 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:34.714 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137374906000005, 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:34.715 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.716 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:34.717 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.719 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.720 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137327706999997, 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.721 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:34.722 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.723 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.724 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:34.725 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137277737000005, 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.726 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.727 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.728 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:34.729 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:34.730 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.137210623000001, 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.731 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.732 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:34.733 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.734 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.735 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.135235537, 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.736 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:34.737 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.738 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.739 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:34.740 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.741 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.135187052000006, 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.742 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.743 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:34.744 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:34.745 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.746 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.135070024000001, 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.747 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:34.748 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.750 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.751 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 89.134978849000007, 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.752 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.753 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.754 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:34.755 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:34.756 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 61.894572001999997, 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:34.757 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.758 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:34.759 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:34.760 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.761 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.762 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:34.763 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:34.765 INFO:tasks.ceph.osd.1.smithi162.stderr:2024-09-16T11:16:34.756+0000 7fb5baa14700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22237) UID: 0 2024-09-16T11:16:34.766 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T11:16:34.766 DEBUG:teuthology.parallel:result is None 2024-09-16T11:16:34.766 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T11:16:34.777 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T11:16:34.785 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T11:16:34.923 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:34.923 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:34.924 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:34.924 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:34.938 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T11:16:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:35.034 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:35.034 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.043 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42188 4.6 4:60b99ad5:::rbd_data.11b5f978907f.000000000000000a:f [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:35.145 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.96218591900001, 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063861163999999998, 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42188 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:35.146 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177949+0000", 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1100000000000001e-07 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.147 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177951+0000", 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7570000000000001e-06 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177957+0000", 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5260000000000001e-06 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.177961+0000", 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0790000000000002e-06 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.148 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241374+0000", 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063413431000000006 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241456+0000", 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1515999999999997e-05 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.241810+0000", 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035444399999999998 2024-09-16T11:16:35.149 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42260 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.96012062700001, 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063950703999999997, 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.150 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42260 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180014+0000", 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.151 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180015+0000", 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6399999999999998e-07 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180016+0000", 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.734e-06 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180022+0000", 2024-09-16T11:16:35.152 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3460000000000001e-06 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180025+0000", 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0869999999999998e-06 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243504+0000", 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063479708999999995 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.153 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243605+0000", 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000100066 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243965+0000", 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000360398 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42261 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.154 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.960106771, 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064069129000000002, 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42261 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.155 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180028+0000", 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8000000000000001e-07 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.156 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180030+0000", 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.871e-06 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180036+0000", 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.276e-06 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180039+0000", 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:35.157 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.243986+0000", 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063947713000000003 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244028+0000", 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1517000000000002e-05 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244097+0000", 2024-09-16T11:16:35.158 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9339999999999997e-05 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42262 4.16 4:69220a3a:::rbd_data.11b570fb0cd.0000000000000001:e [read 122880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.96009289599999, 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064230979999999993, 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.159 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42262 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:35.160 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180042+0000", 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180044+0000", 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.748e-06 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.161 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180049+0000", 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.271e-06 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180052+0000", 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0319999999999999e-06 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244123+0000", 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064070191999999998 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.162 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244213+0000", 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0277000000000003e-05 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244273+0000", 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0108000000000001e-05 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42267 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.163 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.96007912900001, 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064655650999999995, 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42267 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.164 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180056+0000", 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.165 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180058+0000", 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.902e-06 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180063+0000", 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4779999999999999e-06 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180067+0000", 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1130000000000001e-06 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.166 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244331+0000", 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064264204000000005 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244416+0000", 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.4801000000000003e-05 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244711+0000", 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000295801 2024-09-16T11:16:35.167 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42270 4.16 4:6989871f:::rbd_data.11b5f978907f.0000000000000010:f [read 655360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.96003162, 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064996765999999997, 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.168 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42270 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180103+0000", 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.169 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180363+0000", 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025950300000000001 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180371+0000", 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1289999999999996e-06 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180379+0000", 2024-09-16T11:16:35.170 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9070000000000007e-06 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180384+0000", 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9779999999999996e-06 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244778+0000", 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064393802 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.171 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244855+0000", 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7139999999999997e-05 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.245100+0000", 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000245307 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.172 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:42271 4.1e 4:78a7e251:::rbd_data.11b570fb0cd.0000000000000004:e [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.95974580199999, 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064548755999999999, 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 42271 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.173 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.180389+0000", 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184702+0000", 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0043127369999999996 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.174 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184708+0000", 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3470000000000004e-06 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184722+0000", 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3782e-05 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.184728+0000", 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1920000000000002e-06 2024-09-16T11:16:35.175 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244774+0000", 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.060045476 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244847+0000", 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3827999999999999e-05 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.244938+0000", 2024-09-16T11:16:35.176 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0394000000000002e-05 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43830 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.86708198599999, 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063649984000000007, 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.177 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43830 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273053+0000", 2024-09-16T11:16:35.178 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273180+0000", 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012748100000000001 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273186+0000", 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.105e-06 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.179 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273193+0000", 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7410000000000003e-06 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273198+0000", 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5539999999999999e-06 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331723+0000", 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058524964999999998 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.180 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.331798+0000", 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5244e-05 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332124+0000", 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00032654100000000002 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336371+0000", 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.004246283 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.181 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336477+0000", 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106507 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336567+0000", 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9443000000000004e-05 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336703+0000", 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611999999999999 2024-09-16T11:16:35.182 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:43831 4.1e 4:7e74a909:::rbd_object_map.11b5caf806a8:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e54)", 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 170.86693280399999, 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063728673999999999, 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 43831 2024-09-16T11:16:35.183 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.184 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273202+0000", 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1899999999999998e-07 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273204+0000", 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.826e-06 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273208+0000", 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0949999999999998e-06 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.185 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.273211+0000", 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4059999999999999e-06 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332773+0000", 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.059562376 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.332924+0000", 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015061400000000001 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.186 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.333573+0000", 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00064885799999999999 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336811+0000", 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.003238561 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336822+0000", 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0903999999999999e-05 2024-09-16T11:16:35.187 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336888+0000", 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5639e-05 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:44.336931+0000", 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3075999999999999e-05 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.188 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183106 4.1b 4:dd3028ba:::rbd_data.11b5e4cf0723.0000000000000001:28 [read 1099776~2048] snapc 0=[] RETRY=9 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878531584000001, 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063503391000000006, 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183106 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.189 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261603+0000", 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261604+0000", 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9500000000000002e-07 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.190 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261606+0000", 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261612+0000", 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.891e-06 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261615+0000", 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5489999999999998e-06 2024-09-16T11:16:35.191 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324767+0000", 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063151967000000003 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.324869+0000", 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010210299999999999 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325107+0000", 2024-09-16T11:16:35.192 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023737899999999999 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183136 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1036288~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878484623999995, 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063607584999999994, 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.193 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183136 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261650+0000", 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.194 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261651+0000", 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.63e-07 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261653+0000", 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261658+0000", 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9710000000000001e-06 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.195 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261662+0000", 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6569999999999999e-06 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325097+0000", 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063434814000000006 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325169+0000", 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1742999999999994e-05 2024-09-16T11:16:35.196 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325258+0000", 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9117999999999996e-05 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183137 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1042432~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878465931999997, 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063695358999999993, 2024-09-16T11:16:35.197 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183137 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.198 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261669+0000", 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7800000000000002e-07 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261671+0000", 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9719999999999999e-06 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.199 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261677+0000", 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2179999999999996e-06 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261681+0000", 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4999999999999999e-06 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325276+0000", 2024-09-16T11:16:35.200 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063594522000000001 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325316+0000", 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0358000000000002e-05 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325364+0000", 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8411000000000001e-05 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.201 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183139 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1048576~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878449732999997, 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063880934, 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183139 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.202 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261685+0000", 2024-09-16T11:16:35.203 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5400000000000002e-07 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261687+0000", 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8339999999999999e-06 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261693+0000", 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.682e-06 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.204 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261696+0000", 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2270000000000002e-06 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325400+0000", 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063704064000000005 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325471+0000", 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0771000000000004e-05 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.205 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325566+0000", 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5001999999999996e-05 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183141 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1054720~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878434690999995, 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064073827, 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.206 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183141 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.207 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261700+0000", 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261701+0000", 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5600000000000001e-07 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261703+0000", 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9989999999999998e-06 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.208 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261709+0000", 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261712+0000", 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325600+0000", 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.063888405999999995 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.209 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325680+0000", 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9175e-05 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325774+0000", 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4388000000000005e-05 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183143 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1060864~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.210 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878418941000007, 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064303756000000004, 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183143 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:35.211 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261716+0000", 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7399999999999999e-07 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.212 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261718+0000", 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.84e-06 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261724+0000", 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6439999999999997e-06 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261727+0000", 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6270000000000002e-06 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.213 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325803+0000", 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064075220000000002 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.325877+0000", 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.3988999999999999e-05 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326020+0000", 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014306200000000001 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.214 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183144 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1067008~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878403650999999, 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064482752000000004, 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.215 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183144 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261731+0000", 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.216 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261732+0000", 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6600000000000002e-07 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261733+0000", 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.86e-06 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261739+0000", 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8699999999999997e-06 2024-09-16T11:16:35.217 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261743+0000", 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6890000000000002e-06 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326046+0000", 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064302876999999994 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326122+0000", 2024-09-16T11:16:35.218 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5980000000000001e-05 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326214+0000", 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2109999999999997e-05 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183145 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1073152~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878388072999996, 2024-09-16T11:16:35.219 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064647227000000002, 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183145 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:35.220 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261747+0000", 2024-09-16T11:16:35.221 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7e-07 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261749+0000", 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.79e-06 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261755+0000", 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7899999999999996e-06 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.222 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261758+0000", 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4000000000000001e-06 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326239+0000", 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064481187999999995 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326289+0000", 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9344000000000001e-05 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.223 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326394+0000", 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000105345 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:183147 4.3 4:c492962a:::rbd_data.11b5e4cf0723.0000000000000004:28 [read 1079296~2048] snapc 0=[] RETRY=8 ondisk+retry+read+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 90.878373093999997, 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.064744149000000001, 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.224 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 183147 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.225 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.261762+0000", 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268314+0000", 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0065526710000000004 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268320+0000", 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1120000000000004e-06 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.226 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268359+0000", 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9830999999999999e-05 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.268366+0000", 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.241e-06 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326411+0000", 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.058045596999999997 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.227 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326451+0000", 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9855999999999999e-05 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.326506+0000", 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4840999999999999e-05 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:196895 4.7 4:e2573542:::rbd_data.11b51fe98146.000000000000000e:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:35.228 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 62.382883342, 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88411479000000004, 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 196895 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:35.229 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757252+0000", 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757254+0000", 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5560000000000002e-06 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.230 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757265+0000", 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1049999999999999e-05 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757313+0000", 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8248999999999998e-05 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757362+0000", 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8202999999999999e-05 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.231 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757524+0000", 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000162709 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:32.757549+0000", 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5157e-05 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.637970+0000", 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.88042100999999995 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.232 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638143+0000", 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000172196 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.638312+0000", 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016886900000000001 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.640961+0000", 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0026496919999999999 2024-09-16T11:16:35.233 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641209+0000", 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00024805500000000001 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641305+0000", 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5795e-05 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:33.641366+0000", 2024-09-16T11:16:35.234 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1248999999999996e-05 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:204256 4.0 4:0565b8e0:::journal.11b5175acf49:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e206)", 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:16:33.749912+0000", 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 1.3902225859999999, 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89424391400000003, 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.235 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 204256 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.749912+0000", 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.749912+0000", 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.236 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.749913+0000", 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.099e-06 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.749918+0000", 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8110000000000002e-06 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.749931+0000", 2024-09-16T11:16:35.237 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.272e-05 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.749936+0000", 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5249999999999996e-06 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.750006+0000", 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9134000000000006e-05 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:33.750013+0000", 2024-09-16T11:16:35.238 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7479999999999993e-06 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:34.643997+0000", 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.89398408900000004 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:16:34.644156+0000", 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015878800000000001 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.239 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.240 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.240 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.241 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T11:16:35.348 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.348 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:35.348 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:35.348 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.358 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T11:16:35.456 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.457 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:35.457 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:35.457 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:35.457 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.467 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T11:16:35.494 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-16T11:16:35.494 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-09-16T11:16:35.494 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T11:16:35.494 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T11:16:35.497 INFO:tasks.ceph.osd.7:Started 2024-09-16T11:16:35.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:77594 4.1c 4:3a24fc9f:::rbd_object_map.11b578dab209:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 148.07565396499999, 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.018537429, 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.572 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 77594 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492673+0000", 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492677+0000", 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1339999999999997e-06 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492706+0000", 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9085999999999999e-05 2024-09-16T11:16:35.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492741+0000", 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4727999999999997e-05 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492757+0000", 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6194000000000001e-05 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492836+0000", 2024-09-16T11:16:35.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9102000000000003e-05 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.492842+0000", 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5040000000000002e-06 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497588+0000", 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0047458190000000001 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:07.497600+0000", 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2495e-05 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.508905+0000", 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.011304829 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509070+0000", 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016543800000000001 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.577 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 3", 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.509434+0000", 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036350599999999999 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511040+0000", 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001606138 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511146+0000", 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000106475 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511185+0000", 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8813000000000003e-05 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:08.511210+0000", 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5168e-05 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.579 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99538 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.82224116699999, 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11421857000000001, 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.580 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99538 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746086+0000", 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746314+0000", 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000228536 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746321+0000", 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4350000000000002e-06 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746339+0000", 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8218000000000001e-05 2024-09-16T11:16:35.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746348+0000", 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.8810000000000002e-06 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859599+0000", 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113250737 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.859836+0000", 2024-09-16T11:16:35.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000236884 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860304+0000", 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00046887900000000001 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99539 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 718848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:35.584 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.821971851, 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470848, 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99539 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:35.585 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746355+0000", 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746724+0000", 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036914499999999998 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.586 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746735+0000", 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0981e-05 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746744+0000", 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1099999999999992e-06 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746749+0000", 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3989999999999997e-06 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.587 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860340+0000", 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113591498 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860525+0000", 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018486899999999999 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860826+0000", 2024-09-16T11:16:35.588 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00030084600000000001 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99542 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 735232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.821573341, 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114432621, 2024-09-16T11:16:35.589 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99542 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.590 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.746754+0000", 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747012+0000", 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025821900000000002 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747018+0000", 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6799999999999998e-06 2024-09-16T11:16:35.591 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747027+0000", 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3230000000000004e-06 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747038+0000", 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0655999999999999e-05 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.592 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860855+0000", 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.1138171 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.860998+0000", 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014376699999999999 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861186+0000", 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00018787599999999999 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.593 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99544 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 751616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.821284553, 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114531014, 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99544 2024-09-16T11:16:35.594 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.747042+0000", 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.595 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748599+0000", 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001556434 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748605+0000", 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1199999999999999e-06 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.596 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748627+0000", 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2461999999999999e-05 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748634+0000", 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6490000000000002e-06 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861221+0000", 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112586967 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.597 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861347+0000", 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012549800000000001 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.861573+0000", 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000226884 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99548 4.1c 4:385d4ced:::rbd_data.11b578dab209.000000000000000a:1b [read 784384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:35.598 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81958513699999, 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11360105199999999, 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99548 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.599 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748742+0000", 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748743+0000", 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8599999999999998e-07 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748746+0000", 2024-09-16T11:16:35.600 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5109999999999999e-06 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748785+0000", 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9286000000000003e-05 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748814+0000", 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8235999999999999e-05 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.601 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862018+0000", 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113204341 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862171+0000", 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000153358 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862343+0000", 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017163399999999999 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.602 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99550 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.819506821, 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11354982600000001, 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99550 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.603 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748820+0000", 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.604 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748821+0000", 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.9999999999999997e-07 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748824+0000", 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.937e-06 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748842+0000", 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8556e-05 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.605 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748848+0000", 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6960000000000002e-06 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862045+0000", 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113197519 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862198+0000", 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015217799999999999 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.606 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862370+0000", 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017233999999999999 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99552 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.819473082, 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11387768400000001, 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.607 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99552 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.608 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748854+0000", 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9699999999999996e-07 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748857+0000", 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5670000000000002e-06 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.609 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748864+0000", 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9700000000000002e-06 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748880+0000", 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6164e-05 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862412+0000", 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113531668 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.610 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862552+0000", 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014027000000000001 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862732+0000", 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000179548 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99554 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 565248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.611 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81943233699999, 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114470369, 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99554 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:35.612 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.748895+0000", 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752012+0000", 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0031177930000000002 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.613 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752017+0000", 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1980000000000003e-06 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752025+0000", 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6859999999999995e-06 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752029+0000", 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0539999999999996e-06 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.614 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.862785+0000", 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.110755514 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863181+0000", 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00039614300000000001 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.863365+0000", 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183981 2024-09-16T11:16:35.615 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99565 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81623188899999, 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113598809, 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.616 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99565 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.617 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752095+0000", 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.7000000000000001e-07 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752103+0000", 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9689999999999994e-06 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.618 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752107+0000", 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9820000000000002e-06 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752110+0000", 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3549999999999999e-06 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864835+0000", 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.112725612 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.619 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.864979+0000", 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000143969 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865694+0000", 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00071465199999999997 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.620 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99567 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81621383199999, 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11384913300000001, 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99567 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.621 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752113+0000", 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752114+0000", 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5199999999999998e-07 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.622 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752115+0000", 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.24e-06 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752124+0000", 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.1179999999999998e-06 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752136+0000", 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1725999999999999e-05 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.623 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865726+0000", 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11359002 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865815+0000", 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9575999999999998e-05 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.865962+0000", 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014710100000000001 2024-09-16T11:16:35.624 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99569 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81618784299999, 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114068145, 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.625 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99569 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752139+0000", 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.626 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752808+0000", 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00066849300000000004 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752812+0000", 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0219999999999998e-06 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752819+0000", 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4359999999999996e-06 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.627 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752823+0000", 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.659e-06 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866015+0000", 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11319193299999999 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866086+0000", 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1563999999999994e-05 2024-09-16T11:16:35.628 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866207+0000", 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121038 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99571 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 696320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81548941200001, 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113761392, 2024-09-16T11:16:35.629 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99571 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.630 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752838+0000", 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752840+0000", 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.649e-06 2024-09-16T11:16:35.631 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752845+0000", 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4929999999999998e-06 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752848+0000", 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4449999999999999e-06 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866263+0000", 2024-09-16T11:16:35.632 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.113414996 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866394+0000", 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013126699999999999 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866599+0000", 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020498599999999999 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.633 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:99573 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.81547685300001, 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.114125079, 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 99573 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.634 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.752850+0000", 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756455+0000", 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0036045690000000002 2024-09-16T11:16:35.635 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756461+0000", 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2920000000000001e-06 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756474+0000", 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3178e-05 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.756481+0000", 2024-09-16T11:16:35.636 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2019999999999998e-06 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866653+0000", 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.11017120800000001 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866792+0000", 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013911699999999999 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:21.866975+0000", 2024-09-16T11:16:35.637 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000183513 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:164922 4.7 4:e254db35:::rbd_object_map.11b596c7e9d.0000000000000029:head [writefull 0~107] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 100.90580110000001, 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7976434029999999, 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.638 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 164922 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662526+0000", 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.639 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662528+0000", 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2979999999999999e-06 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662537+0000", 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0969999999999995e-06 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662559+0000", 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2133e-05 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.640 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662568+0000", 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1899999999999995e-06 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662632+0000", 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.4172999999999997e-05 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:54.662641+0000", 2024-09-16T11:16:35.641 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3980000000000005e-06 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431389+0000", 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.76874769399999998 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:55.431413+0000", 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4273999999999998e-05 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.642 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458124+0000", 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0267112279999999 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458277+0000", 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015295200000000001 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.458409+0000", 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013202799999999999 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.643 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.459194+0000", 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00078427200000000003 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460008+0000", 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00081413500000000001 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460049+0000", 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0926000000000001e-05 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.644 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:56.460169+0000", 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000120605 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185792 4.2 4:47a23f8a:::rbd_data.11b596c7e9d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.223375594999993, 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.36211669800000001, 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.645 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185792 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.646 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344951+0000", 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.344953+0000", 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.6980000000000001e-06 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.647 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345231+0000", 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027838400000000001 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345249+0000", 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7201999999999999e-05 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345255+0000", 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8190000000000003e-06 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.648 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345305+0000", 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9845000000000001e-05 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.345312+0000", 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1289999999999998e-06 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.702947+0000", 2024-09-16T11:16:35.649 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35763499799999998 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703118+0000", 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00017089000000000001 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703319+0000", 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000200282 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.650 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706900+0000", 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0035817890000000002 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706972+0000", 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1267999999999996e-05 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707028+0000", 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5983000000000003e-05 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.651 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707068+0000", 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0411000000000003e-05 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185800 4.2 4:45b13b1d:::rbd_data.11b596c7e9d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.220510701999999, 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35940005000000003, 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.652 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185800 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.653 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347816+0000", 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347818+0000", 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347823+0000", 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4219999999999999e-06 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.654 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347835+0000", 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2189e-05 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347841+0000", 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0589999999999999e-06 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347954+0000", 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000112581 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.655 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.347960+0000", 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5769999999999999e-06 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.703864+0000", 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35590402799999998 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704023+0000", 2024-09-16T11:16:35.656 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015823399999999999 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704205+0000", 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000182726 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706906+0000", 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0027001769999999998 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.657 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707122+0000", 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000216281 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707179+0000", 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7172000000000001e-05 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707216+0000", 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7277999999999998e-05 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.658 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185820 4.2 4:41b775c9:::rbd_data.11b596c7e9d.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.211464785999993, 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.35060028999999998, 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.659 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185820 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356862+0000", 2024-09-16T11:16:35.660 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.356863+0000", 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5899999999999995e-07 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357312+0000", 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00044930600000000001 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.661 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357324+0000", 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2027e-05 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357331+0000", 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9099999999999999e-06 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357472+0000", 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014097400000000001 2024-09-16T11:16:35.662 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.357479+0000", 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1210000000000001e-06 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704582+0000", 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.34710309499999997 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704682+0000", 2024-09-16T11:16:35.663 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9437000000000004e-05 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.704875+0000", 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00019310799999999999 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706933+0000", 2024-09-16T11:16:35.664 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002058051 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707412+0000", 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047933399999999999 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.665 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707442+0000", 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9669e-05 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707462+0000", 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0499000000000001e-05 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185848 4.2 4:400a0a99:::rbd_data.11b596c7e9d.0000000000000032:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.666 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.154049280999999, 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29572977700000003, 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185848 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:35.667 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414278+0000", 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414279+0000", 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.542e-06 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.668 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414286+0000", 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1219999999999997e-06 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414304+0000", 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7365999999999999e-05 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414312+0000", 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3920000000000003e-06 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.669 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414374+0000", 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1721000000000001e-05 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.414381+0000", 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4059999999999999e-06 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705480+0000", 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.29109863499999999 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.670 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705741+0000", 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000260618 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.705946+0000", 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00020521299999999999 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.707516+0000", 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001570583 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.671 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709914+0000", 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.002397672 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.709970+0000", 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6326999999999998e-05 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710007+0000", 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.718e-05 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.672 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:185858 4.2 4:40797b13:::rbd_data.11b596c7e9d.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 91.107790116000004, 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24955571700000001, 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 185858 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.673 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460537+0000", 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460539+0000", 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8190000000000001e-06 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.674 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460546+0000", 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9249999999999998e-06 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460564+0000", 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8162999999999999e-05 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460572+0000", 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6449999999999994e-06 2024-09-16T11:16:35.675 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460630+0000", 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7509000000000001e-05 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.460641+0000", 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1432e-05 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:35.676 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706222+0000", 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.24558041 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706304+0000", 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2477000000000003e-05 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 4", 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.706400+0000", 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.5612999999999994e-05 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.677 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.708003+0000", 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016026840000000001 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710020+0000", 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0020170629999999999 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710064+0000", 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3968999999999999e-05 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:35.678 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:04.710093+0000", 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9008000000000001e-05 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:35.679 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:no valid command found; 10 closest matches: 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:0 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:1 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:2 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:abort 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:assert 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:config diff 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:config diff get 2024-09-16T11:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:config get 2024-09-16T11:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:config help [] 2024-09-16T11:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:config set ... 2024-09-16T11:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:admin_socket: invalid command 2024-09-16T11:16:35.682 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T11:16:35.682 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-16T11:16:35.792 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:35.792 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:35.793 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.802 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T11:16:35.860 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:16:35.855+0000 7fc3ad3e5540 -1 Falling back to public interface 2024-09-16T11:16:35.904 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:35.904 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:35.905 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:35.905 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:35.905 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:35.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103259 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1177600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.020 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93905685799999, 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213981219999999, 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103259 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.021 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076955+0000", 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076956+0000", 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3700000000000001e-07 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.022 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076957+0000", 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.728e-06 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076963+0000", 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6019999999999999e-06 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076966+0000", 2024-09-16T11:16:36.023 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.19e-06 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077442+0000", 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00047551499999999998 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077446+0000", 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.121e-06 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.024 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098219+0000", 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0207728190000001 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098288+0000", 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8948000000000005e-05 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098353+0000", 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5862000000000001e-05 2024-09-16T11:16:36.025 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103261 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1193984~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93904240000001, 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021502275, 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.026 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103261 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.027 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076970+0000", 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1600000000000002e-07 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076972+0000", 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.148e-06 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.028 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076978+0000", 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7969999999999997e-06 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076983+0000", 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7310000000000001e-06 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.029 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077464+0000", 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048158599999999999 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077468+0000", 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0860000000000004e-06 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098369+0000", 2024-09-16T11:16:36.030 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0209002549999999 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098409+0000", 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0593999999999998e-05 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098472+0000", 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2762000000000007e-05 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.031 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103264 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1210368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.9390252, 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217437540000001, 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103264 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.032 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076987+0000", 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.77e-07 2024-09-16T11:16:36.033 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076989+0000", 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9209999999999999e-06 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076995+0000", 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5509999999999999e-06 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.034 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.076998+0000", 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3560000000000001e-06 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077487+0000", 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00048846899999999995 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077491+0000", 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7269999999999998e-06 2024-09-16T11:16:36.035 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098486+0000", 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.020994749 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098567+0000", 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0957000000000004e-05 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098731+0000", 2024-09-16T11:16:36.036 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000163647 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103268 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1226752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93901167999999, 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219683980000001, 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.037 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103268 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:36.038 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077001+0000", 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5600000000000002e-07 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077002+0000", 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.322e-06 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.039 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077005+0000", 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8389999999999998e-06 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077007+0000", 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.17e-06 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077508+0000", 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00050080999999999997 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.040 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077512+0000", 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.16e-06 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098761+0000", 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0212493739999999 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098847+0000", 2024-09-16T11:16:36.041 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.5513000000000006e-05 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098969+0000", 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000121954 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103270 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1243136~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.939003138, 2024-09-16T11:16:36.042 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222565669999999, 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103270 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.043 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077009+0000", 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4600000000000001e-07 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077010+0000", 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.114e-06 2024-09-16T11:16:36.044 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077013+0000", 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9129999999999999e-06 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077016+0000", 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2579999999999998e-06 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077530+0000", 2024-09-16T11:16:36.045 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051464700000000002 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077535+0000", 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7439999999999998e-06 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.098991+0000", 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021455556 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.046 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099148+0000", 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015708200000000001 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099266+0000", 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000118007 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103273 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1259520~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:36.047 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93899299399999, 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224046600000001, 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103273 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:36.048 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077019+0000", 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077020+0000", 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7599999999999998e-07 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.049 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077022+0000", 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2110000000000001e-06 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077027+0000", 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3970000000000002e-06 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077029+0000", 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0339999999999999e-06 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.050 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077553+0000", 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052374300000000002 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077559+0000", 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.127e-06 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099300+0000", 2024-09-16T11:16:36.051 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0217406769999999 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099355+0000", 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5247999999999997e-05 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099424+0000", 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8847e-05 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.052 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103275 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1275904~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93897948700001, 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02254114, 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103275 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.053 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077033+0000", 2024-09-16T11:16:36.054 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5699999999999998e-07 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077035+0000", 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5999999999999999e-06 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077039+0000", 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7319999999999998e-06 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.055 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077043+0000", 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2100000000000002e-06 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077578+0000", 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053499500000000002 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077582+0000", 2024-09-16T11:16:36.056 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4070000000000003e-06 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099454+0000", 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021872267 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099503+0000", 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8344000000000003e-05 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.057 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099574+0000", 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1228000000000003e-05 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103278 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1292288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.938965895, 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0226656350000001, 2024-09-16T11:16:36.058 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103278 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.059 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077046+0000", 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077047+0000", 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7500000000000001e-07 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077057+0000", 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0813e-05 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.060 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077063+0000", 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5570000000000003e-06 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7179999999999998e-06 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077603+0000", 2024-09-16T11:16:36.061 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053583700000000005 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077607+0000", 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2130000000000002e-06 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099605+0000", 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0219984390000001 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.062 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099650+0000", 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4248000000000001e-05 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099712+0000", 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2434999999999994e-05 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103280 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1308672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:36.063 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.938941613, 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0227972830000001, 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103280 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.064 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077071+0000", 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.065 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077073+0000", 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0020000000000001e-06 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077079+0000", 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5350000000000004e-06 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077082+0000", 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9689999999999999e-06 2024-09-16T11:16:36.066 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077628+0000", 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00054627899999999999 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077632+0000", 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1760000000000003e-06 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099743+0000", 2024-09-16T11:16:36.067 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0221106630000001 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099801+0000", 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8198000000000001e-05 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099868+0000", 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7027999999999996e-05 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.068 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103946 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1310720~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.938907452, 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0230016829999999, 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103946 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.069 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.070 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077105+0000", 2024-09-16T11:16:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.5499999999999998e-07 2024-09-16T11:16:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.071 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077108+0000", 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2079999999999999e-06 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077113+0000", 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8239999999999996e-06 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.072 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077118+0000", 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999999e-06 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077650+0000", 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00053285500000000005 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077655+0000", 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4789999999999998e-06 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.073 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099900+0000", 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0222450089999999 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.099982+0000", 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.2076999999999994e-05 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100106+0000", 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000124416 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.074 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103948 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1327104~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93887405500001, 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0231804929999999, 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103948 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.075 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077138+0000", 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077139+0000", 2024-09-16T11:16:36.076 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2100000000000002e-07 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077141+0000", 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.0619999999999999e-06 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077147+0000", 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077156+0000", 2024-09-16T11:16:36.077 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3400000000000004e-06 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077670+0000", 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051364399999999995 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077674+0000", 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0119999999999999e-06 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.078 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100140+0000", 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0224666950000001 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100197+0000", 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6416999999999999e-05 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100319+0000", 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012173899999999999 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.079 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103950 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1343488~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93885141000001, 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.02336815, 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103950 2024-09-16T11:16:36.080 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077161+0000", 2024-09-16T11:16:36.081 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9900000000000001e-07 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077163+0000", 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9300000000000002e-06 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077169+0000", 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7690000000000002e-06 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.082 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077172+0000", 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.3730000000000001e-06 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077691+0000", 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00051826400000000005 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077694+0000", 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.788e-06 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.083 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100343+0000", 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022649081 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100393+0000", 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9434999999999997e-05 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100529+0000", 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00013611100000000001 2024-09-16T11:16:36.084 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103952 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1359872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93883583499999, 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0235666109999999, 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.085 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103952 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077176+0000", 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.086 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077177+0000", 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8799999999999998e-07 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077178+0000", 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7439999999999999e-06 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077184+0000", 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8019999999999997e-06 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.087 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077188+0000", 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4400000000000001e-06 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077713+0000", 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00052552999999999996 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077717+0000", 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8600000000000003e-06 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.088 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100562+0000", 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0228444189999999 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100641+0000", 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.9703000000000006e-05 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100743+0000", 2024-09-16T11:16:36.089 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00010172500000000001 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103954 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.938818801, 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.023738939, 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.090 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103954 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077193+0000", 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.091 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077194+0000", 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.3599999999999999e-07 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077196+0000", 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7680000000000001e-06 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077203+0000", 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:36.092 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077207+0000", 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.8929999999999998e-06 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084167+0000", 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0069604690000000004 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.093 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084190+0000", 2024-09-16T11:16:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2847999999999999e-05 2024-09-16T11:16:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.094 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100766+0000", 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0165760779999999 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100835+0000", 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8603999999999996e-05 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.095 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.100932+0000", 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.7409000000000002e-05 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103965 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1474560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93518583700001, 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0213393879999999, 2024-09-16T11:16:36.096 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103965 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.097 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080826+0000", 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080827+0000", 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2599999999999998e-07 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080844+0000", 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7455000000000001e-05 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.098 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080851+0000", 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.1010000000000003e-06 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080856+0000", 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.0529999999999997e-06 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084740+0000", 2024-09-16T11:16:36.099 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038831130000000001 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084744+0000", 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.1969999999999998e-06 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.101986+0000", 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0172423159999999 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.100 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102039+0000", 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.3421000000000002e-05 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102166+0000", 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012630599999999999 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103967 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1490944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:36.101 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93515149500001, 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021453749, 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103967 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.102 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080861+0000", 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.7999999999999996e-07 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.103 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080863+0000", 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1229999999999998e-06 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080869+0000", 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0850000000000002e-06 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080873+0000", 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4510000000000001e-06 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.104 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084763+0000", 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038902889999999999 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084767+0000", 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.228e-06 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102181+0000", 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017413975 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.105 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102239+0000", 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7887000000000003e-05 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102314+0000", 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.5230999999999999e-05 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103969 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.106 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93513554899999, 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021633381, 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103969 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.107 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.108 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080877+0000", 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9700000000000002e-07 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080879+0000", 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9069999999999999e-06 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080885+0000", 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6629999999999998e-06 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.109 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080888+0000", 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6440000000000002e-06 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084787+0000", 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0038992160000000001 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084792+0000", 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.4850000000000003e-06 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.110 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102337+0000", 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.017544655 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102407+0000", 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0313000000000003e-05 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102510+0000", 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000103101 2024-09-16T11:16:36.111 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103970 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93511993800001, 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021782722, 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.112 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103970 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080892+0000", 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.113 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080893+0000", 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6899999999999998e-07 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080894+0000", 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.638e-06 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080900+0000", 2024-09-16T11:16:36.114 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6749999999999999e-06 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080903+0000", 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.117e-06 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084812+0000", 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039091100000000004 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084816+0000", 2024-09-16T11:16:36.115 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.2910000000000001e-06 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102525+0000", 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0177084810000001 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102579+0000", 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.4029000000000001e-05 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.116 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102675+0000", 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6012000000000002e-05 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103971 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1540096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93510484999999, 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.021964952, 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.117 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103971 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.118 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080907+0000", 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080908+0000", 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7300000000000002e-07 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080910+0000", 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8610000000000001e-06 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.119 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080915+0000", 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7069999999999997e-06 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080919+0000", 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.2509999999999999e-06 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084836+0000", 2024-09-16T11:16:36.120 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0039176640000000004 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.084841+0000", 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5709999999999999e-06 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102703+0000", 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0178623600000001 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.121 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102753+0000", 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9663999999999997e-05 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102872+0000", 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00011950100000000001 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:103973 4.4 4:20ea52f4:::rbd_data.11b578dab209.000000000000000c:1b [read 1556480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:36.122 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 133.93509004800001, 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.022156142, 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 103973 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.123 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080922+0000", 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080923+0000", 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9299999999999999e-07 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080924+0000", 2024-09-16T11:16:36.124 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.719e-06 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080929+0000", 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.147e-06 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.080933+0000", 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1839999999999999e-06 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085390+0000", 2024-09-16T11:16:36.125 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0044577460000000003 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.085396+0000", 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0120000000000002e-06 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102903+0000", 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.01750674 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.102976+0000", 2024-09-16T11:16:36.126 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2633999999999996e-05 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.103078+0000", 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000102567 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.127 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:36.129 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T11:16:36.245 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:36.245 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:36.245 INFO:teuthology.orchestra.run.smithi162.stdout: "num_ops": 0 2024-09-16T11:16:36.245 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:36.255 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T11:16:36.362 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [], 2024-09-16T11:16:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "complaint_time": 30, 2024-09-16T11:16:36.362 INFO:teuthology.orchestra.run.smithi162.stdout: "num_blocked_ops": 0 2024-09-16T11:16:36.363 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:36.372 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T11:16:36.480 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "size": 20, 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 600, 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "ops": [ 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49777 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.343749774, 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050712833999999998, 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.481 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49777 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.482 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133314+0000", 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.6899999999999997e-07 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133317+0000", 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9950000000000001e-06 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.483 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133325+0000", 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1370000000000002e-06 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133329+0000", 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7799999999999998e-06 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183661+0000", 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050332189999999999 2024-09-16T11:16:36.484 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.183752+0000", 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.0651999999999999e-05 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184026+0000", 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00027451100000000001 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.485 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49778 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1482752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.343731386, 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051249296, 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49778 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.486 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133332+0000", 2024-09-16T11:16:36.487 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.65e-07 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133334+0000", 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.376e-06 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133337+0000", 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.794e-06 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.488 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133341+0000", 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0910000000000003e-06 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184122+0000", 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050780817999999998 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184215+0000", 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.2230000000000003e-05 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184581+0000", 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00036672199999999999 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49779 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.34343062900001, 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051441635999999999, 2024-09-16T11:16:36.490 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49779 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.491 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133633+0000", 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133634+0000", 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.085e-06 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133639+0000", 2024-09-16T11:16:36.492 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2449999999999998e-06 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133653+0000", 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3669000000000001e-05 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133658+0000", 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8400000000000002e-06 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.493 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184614+0000", 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050956571999999999 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.184721+0000", 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000107104 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185074+0000", 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00035312100000000002 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.494 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49780 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1613824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.34340092400001, 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051736356999999997, 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49780 2024-09-16T11:16:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133662+0000", 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.496 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133663+0000", 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133665+0000", 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9479999999999998e-06 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133671+0000", 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1360000000000002e-06 2024-09-16T11:16:36.497 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133673+0000", 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.424e-06 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185105+0000", 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051431216000000002 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.498 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185162+0000", 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.7068000000000003e-05 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185399+0000", 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00023711199999999999 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49781 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1679360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:36.499 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.34338752599999, 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052096135000000002, 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49781 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.500 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133676+0000", 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.501 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133677+0000", 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.277e-06 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133682+0000", 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8969999999999999e-06 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133685+0000", 2024-09-16T11:16:36.502 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.9399999999999998e-06 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185427+0000", 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051741654999999998 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185510+0000", 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3456999999999997e-05 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.503 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185772+0000", 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026161700000000001 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49782 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1744896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.343373722, 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052443893999999998, 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.504 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49782 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:36.505 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133690+0000", 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8000000000000002e-07 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133691+0000", 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.33e-06 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.506 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133696+0000", 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1359999999999996e-06 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.133699+0000", 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1610000000000001e-06 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185801+0000", 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052102383000000002 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.507 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.185880+0000", 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.8818000000000003e-05 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186134+0000", 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025378599999999999 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.508 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:49783 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.34299872299999, 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052478359000000002, 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 49783 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.509 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134065+0000", 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134066+0000", 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.9600000000000008e-07 2024-09-16T11:16:36.510 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134070+0000", 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.8189999999999999e-06 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134084+0000", 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3906999999999999e-05 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.134091+0000", 2024-09-16T11:16:36.511 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.6630000000000004e-06 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186153+0000", 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.052062366999999998 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186231+0000", 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.7353000000000003e-05 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.512 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.186543+0000", 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00031225399999999998 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50036 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.32945469200001, 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050574362999999997, 2024-09-16T11:16:36.513 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50036 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.514 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147609+0000", 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.1100000000000002e-07 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147616+0000", 2024-09-16T11:16:36.515 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7209999999999997e-06 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147622+0000", 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5919999999999997e-06 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147626+0000", 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.9380000000000002e-06 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.516 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197593+0000", 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.049966623000000002 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.197691+0000", 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.8459e-05 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198183+0000", 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00049171900000000005 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.517 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50037 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.329433118, 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051067524000000003, 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50037 2024-09-16T11:16:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147630+0000", 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.519 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147631+0000", 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9599999999999999e-07 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147633+0000", 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2120000000000002e-06 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147639+0000", 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1029999999999998e-06 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.520 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147643+0000", 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7450000000000001e-06 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198230+0000", 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050586919000000001 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198326+0000", 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6656000000000004e-05 2024-09-16T11:16:36.521 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198698+0000", 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00037139300000000003 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50038 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.329416619, 2024-09-16T11:16:36.522 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051458049999999998, 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50038 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.523 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147647+0000", 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.6199999999999998e-07 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.524 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147650+0000", 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3029999999999998e-06 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147655+0000", 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.8969999999999996e-06 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147659+0000", 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.484e-06 2024-09-16T11:16:36.525 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198736+0000", 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051077035 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.198817+0000", 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.0943000000000001e-05 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.526 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199105+0000", 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00028792599999999998 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50039 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.32940054299999, 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051805904999999999, 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.527 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50039 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.528 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147663+0000", 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.0400000000000002e-07 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147665+0000", 2024-09-16T11:16:36.529 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1600000000000001e-06 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147672+0000", 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.1380000000000004e-06 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.147675+0000", 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.5580000000000001e-06 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.530 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199154+0000", 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051478536999999998 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199236+0000", 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1940999999999994e-05 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199469+0000", 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000233167 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.531 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50085 4.0 4:01c04e59:::rbd_data.11b5caf806a8.000000000000000c:10 [read 49152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.32901696900001, 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051253979999999998, 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.532 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50085 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148046+0000", 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.533 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148047+0000", 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0550000000000001e-06 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148053+0000", 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.1569999999999999e-06 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148063+0000", 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0225e-05 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.534 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.148070+0000", 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7800000000000003e-06 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199135+0000", 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051065747000000002 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199207+0000", 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.2002999999999995e-05 2024-09-16T11:16:36.535 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.199300+0000", 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.3012999999999999e-05 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50294 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.316926591, 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050913201999999998, 2024-09-16T11:16:36.536 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50294 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.537 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160137+0000", 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4200000000000002e-07 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160138+0000", 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2249999999999999e-06 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.538 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160141+0000", 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.779e-06 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160143+0000", 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.815e-06 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210699+0000", 2024-09-16T11:16:36.539 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050555729000000001 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.210785+0000", 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.6539000000000006e-05 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211050+0000", 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000264873 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.540 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:50295 4.10 4:0d7ec2fe:::rbd_data.11b5eaebe1b2.0000000000000001:11 [read 1826816~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 168.31691856899999, 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.051340113, 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 50295 2024-09-16T11:16:36.541 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.542 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160145+0000", 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.3999999999999998e-07 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160146+0000", 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1850000000000001e-06 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160149+0000", 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.8909999999999999e-06 2024-09-16T11:16:36.543 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.160151+0000", 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.7460000000000001e-06 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211150+0000", 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.050999210000000003 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.544 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211231+0000", 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.1429999999999998e-05 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:13:48.211485+0000", 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00025341099999999999 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:94661 4.11 4:8d6a55b5:::rbd_data.11b56d1517d8.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-16T11:16:36.545 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 139.63343968999999, 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.192977391, 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 94661 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:36.546 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843624+0000", 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.843625+0000", 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2270000000000001e-06 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.547 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865068+0000", 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.021443192999999999 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865102+0000", 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.4387999999999997e-05 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865111+0000", 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9430000000000006e-06 2024-09-16T11:16:36.548 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865164+0000", 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 5.2537999999999997e-05 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.865289+0000", 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012541599999999999 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.866924+0000", 2024-09-16T11:16:36.549 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0016345960000000001 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:16.938347+0000", 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.07142271 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036331+0000", 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.097983923000000001 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.550 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036492+0000", 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016154 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:17.036601+0000", 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.000108917 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:104004 4.19 4:9e9f3f03:::rbd_header.11b5cb71539e:head [watch reconnect cookie 140364901035248 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T11:16:36.551 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 134.40002133900001, 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.014427835, 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 104004 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:36.552 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077042+0000", 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077044+0000", 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.733e-06 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.553 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077050+0000", 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2129999999999996e-06 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077067+0000", 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.681e-05 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.556 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.077074+0000", 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.8560000000000001e-06 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078134+0000", 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0010600340000000001 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op must wait for map", 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.078146+0000", 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2571999999999999e-05 2024-09-16T11:16:36.557 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079638+0000", 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001491846 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:22.079648+0000", 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.4499999999999993e-06 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.558 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089347+0000", 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.0096995520000001 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.089697+0000", 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00034997000000000001 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:23.091470+0000", 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.001772799 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.559 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:132644 4.1a 4:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 125.013627966, 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.091312387999999994, 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "started", 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.560 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 132644 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463435+0000", 2024-09-16T11:16:36.561 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463437+0000", 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.8500000000000001e-06 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463444+0000", 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.3799999999999999e-06 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.562 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463459+0000", 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5676999999999999e-05 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463466+0000", 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.9480000000000002e-06 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463543+0000", 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.6873999999999996e-05 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.563 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.463554+0000", 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1355999999999999e-05 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554319+0000", 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.090764687999999996 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554481+0000", 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00016179500000000001 2024-09-16T11:16:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:31.554748+0000", 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00026682000000000002 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:133220 4.14 4:2a0a0ac0:::rbd_data.11b5eb152d9.0000000000000004:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 123.062023318, 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.053851481, 2024-09-16T11:16:36.565 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 133220 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.566 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415040+0000", 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415043+0000", 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.5129999999999999e-06 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415055+0000", 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.1956999999999999e-05 2024-09-16T11:16:36.567 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415091+0000", 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.6260000000000002e-05 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415110+0000", 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9349999999999999e-05 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415177+0000", 2024-09-16T11:16:36.568 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.7179000000000002e-05 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.415274+0000", 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 9.6243000000000007e-05 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.424627+0000", 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.009353933 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468770+0000", 2024-09-16T11:16:36.569 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.044142979999999998 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468786+0000", 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.5917999999999999e-05 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468851+0000", 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5128000000000004e-05 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.570 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:33.468892+0000", 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.002e-05 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:144765 4.10 4:09fec08c:::image_client.0-clone33.rbd:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97)", 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 115.926226386, 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.2263345490000002, 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "reached pg", 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.571 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 144765 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550837+0000", 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.572 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550838+0000", 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.3540000000000001e-06 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550845+0000", 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.5799999999999997e-06 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550864+0000", 2024-09-16T11:16:36.573 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.9046999999999999e-05 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550872+0000", 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.3960000000000006e-06 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550962+0000", 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 8.9697999999999994e-05 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.574 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:40.550974+0000", 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2289999999999999e-05 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513833+0000", 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.96285863299999996 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:41.513863+0000", 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.0332999999999999e-05 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.575 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777015+0000", 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 1.2631512060000001 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:14:42.777172+0000", 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00015701199999999999 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.576 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "description": "osd_op(client.4533.0:198583 4.10 4:0b7b6f02:::rbd_id.image_client.0-clone30:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "initiated_at": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "age": 48.873502467999998, 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.93358903599999998, 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "type_data": { 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_info": { 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client": "client.4533", 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "tid": 198583 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.577 INFO:teuthology.orchestra.run.smithi162.stdout: "events": [ 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "initiated", 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "header_read", 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603561+0000", 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "throttled", 2024-09-16T11:16:36.578 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603563+0000", 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.1210000000000001e-06 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "all_read", 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603569+0000", 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2269999999999998e-06 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "dispatched", 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603593+0000", 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 2.4256e-05 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.579 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603600+0000", 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.0780000000000001e-06 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603662+0000", 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 6.2656000000000005e-05 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:47.603669+0000", 2024-09-16T11:16:36.580 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 7.0559999999999999e-06 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "reached_pg", 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533490+0000", 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.92982107400000003 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "started", 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533633+0000", 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00014247500000000001 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.581 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "waiting for subops from 6", 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.533758+0000", 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00012511 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "op_commit", 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.536659+0000", 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.0029015 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537063+0000", 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 0.00040358599999999999 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.582 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "commit_sent", 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537113+0000", 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 4.9722000000000003e-05 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: { 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: "event": "done", 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: "time": "2024-09-16T11:15:48.537150+0000", 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: "duration": 3.7175000000000002e-05 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-16T11:16:36.583 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:16:36.585 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T11:16:36.656 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T11:16:36.696 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:36.696 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:36.696 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:36.696 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:36.706 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T11:16:36.807 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:36.808 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:36.808 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:36.808 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:36.808 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:36.817 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102680 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1050624~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.917 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.838577679, 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080266865, 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102680 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.918 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074010+0000", 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074011+0000", 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4300000000000003e-07 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.919 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074013+0000", 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0090000000000002e-06 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074021+0000", 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9559999999999997e-06 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074025+0000", 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9349999999999996e-06 2024-09-16T11:16:36.920 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083159+0000", 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091343550000000003 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083164+0000", 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7049999999999998e-06 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.921 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.153954+0000", 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0707900459999999 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154094+0000", 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000139999 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.922 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154277+0000", 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018331700000000001 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102682 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1067008~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.838536917, 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080457504, 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.923 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102682 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074051+0000", 2024-09-16T11:16:36.924 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074052+0000", 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2099999999999996e-07 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074055+0000", 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0000000000000001e-06 2024-09-16T11:16:36.925 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074063+0000", 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.3639999999999999e-06 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074069+0000", 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8610000000000003e-06 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.926 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083189+0000", 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091196469999999998 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083192+0000", 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3160000000000001e-06 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154303+0000", 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071111436 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.927 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154386+0000", 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2604999999999999e-05 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154509+0000", 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012265400000000001 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.928 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102683 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1083392~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83850611899999, 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080716676, 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102683 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.929 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074082+0000", 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074083+0000", 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5700000000000002e-07 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.930 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074085+0000", 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.469e-06 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074092+0000", 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.72e-06 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074096+0000", 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3470000000000001e-06 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.931 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083218+0000", 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091220109999999993 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083222+0000", 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5990000000000002e-06 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154580+0000", 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.071358526 2024-09-16T11:16:36.932 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154667+0000", 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6495000000000004e-05 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154799+0000", 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013195200000000001 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.933 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102685 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1099776~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.838487531, 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.080936108, 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102685 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.934 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074101+0000", 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9099999999999999e-07 2024-09-16T11:16:36.935 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074103+0000", 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1399999999999998e-06 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074109+0000", 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0990000000000004e-06 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074113+0000", 2024-09-16T11:16:36.936 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.941e-06 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083247+0000", 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091336339999999999 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083250+0000", 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.937 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154836+0000", 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0715853529999999 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.154916+0000", 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0696000000000001e-05 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155037+0000", 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012047100000000001 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.938 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102687 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1116160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.838469479, 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081153571, 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102687 2024-09-16T11:16:36.939 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.940 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074119+0000", 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3700000000000001e-07 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074121+0000", 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9429999999999999e-06 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074127+0000", 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1630000000000001e-06 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.941 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074132+0000", 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6269999999999999e-06 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083279+0000", 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091469800000000007 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083282+0000", 2024-09-16T11:16:36.942 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6200000000000001e-06 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155064+0000", 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0717814080000001 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155155+0000", 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1410999999999996e-05 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.943 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155272+0000", 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000116982 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102689 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1132544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83844955199999, 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0813732039999999, 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.944 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102689 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.945 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074139+0000", 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5700000000000002e-07 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074141+0000", 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1330000000000002e-06 2024-09-16T11:16:36.946 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074148+0000", 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8199999999999999e-06 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074153+0000", 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7389999999999999e-06 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.947 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083308+0000", 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0091552809999999995 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083312+0000", 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.399e-06 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155298+0000", 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0719868539999999 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.948 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155393+0000", 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.4365999999999997e-05 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155512+0000", 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000118955 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.949 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102691 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1148928~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83843071300001, 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081570009, 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102691 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.950 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074157+0000", 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074314+0000", 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015694999999999999 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.951 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074331+0000", 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6237000000000001e-05 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074338+0000", 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8890000000000002e-06 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.952 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074344+0000", 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3000000000000001e-06 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083338+0000", 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.008994508 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083342+0000", 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.653e-06 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.953 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155546+0000", 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0722045579999999 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155625+0000", 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8781000000000003e-05 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155727+0000", 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010213300000000001 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.954 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102693 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1165312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83823876400001, 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0816311430000001, 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102693 2024-09-16T11:16:36.955 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074349+0000", 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.956 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074350+0000", 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5599999999999995e-07 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074352+0000", 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.379e-06 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074358+0000", 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9549999999999997e-06 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.957 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074363+0000", 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.203e-06 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083366+0000", 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0090026740000000004 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083369+0000", 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3510000000000002e-06 2024-09-16T11:16:36.958 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155750+0000", 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0723809559999999 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155855+0000", 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010418 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.959 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.155980+0000", 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00012588899999999999 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102695 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1181696~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83821948400001, 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818567299999999, 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.960 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102695 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.961 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.074369+0000", 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077735+0000", 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0033668259999999998 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077740+0000", 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.34e-06 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.962 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077750+0000", 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0515e-05 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077756+0000", 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.1449999999999996e-06 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083398+0000", 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0056412750000000003 2024-09-16T11:16:36.963 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083401+0000", 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5420000000000001e-06 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156029+0000", 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0726279869999999 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.964 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156123+0000", 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3474999999999995e-05 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.156225+0000", 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102625 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102708 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1280000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.965 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83472791599999, 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0802148330000001, 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102708 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:36.966 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077860+0000", 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8099999999999999e-07 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.967 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077862+0000", 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6589999999999999e-06 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077866+0000", 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5559999999999999e-06 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077868+0000", 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0839999999999999e-06 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.968 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083573+0000", 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005705407 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083579+0000", 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.502e-06 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.157930+0000", 2024-09-16T11:16:36.969 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074351362 2024-09-16T11:16:36.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.970 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158006+0000", 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.6316000000000006e-05 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.972 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158075+0000", 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8665999999999996e-05 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102710 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1296384~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.834717845, 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0806448230000001, 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.973 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102710 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077870+0000", 2024-09-16T11:16:36.974 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077871+0000", 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7700000000000001e-07 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077872+0000", 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3230000000000001e-06 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.975 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077875+0000", 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.813e-06 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077877+0000", 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1799999999999999e-06 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083605+0000", 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005727902 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.976 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083608+0000", 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3830000000000001e-06 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158132+0000", 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.07452383 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158268+0000", 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013641599999999999 2024-09-16T11:16:36.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158515+0000", 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024669900000000002 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102737 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1052672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83468954200001, 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.08073928, 2024-09-16T11:16:36.978 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102737 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.979 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077899+0000", 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.96e-07 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077900+0000", 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.491e-06 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.980 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077908+0000", 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4340000000000003e-06 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077910+0000", 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3350000000000001e-06 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083636+0000", 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005726208 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.981 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083640+0000", 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6509999999999999e-06 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158502+0000", 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0748617840000001 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158553+0000", 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1304999999999998e-05 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.982 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158638+0000", 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4776000000000003e-05 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102739 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1069056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.834675613, 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0812286740000001, 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.983 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102739 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.984 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077912+0000", 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077913+0000", 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077914+0000", 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.15e-06 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.985 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077917+0000", 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.954e-06 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077919+0000", 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1670000000000002e-06 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083664+0000", 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005744898 2024-09-16T11:16:36.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083667+0000", 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4039999999999999e-06 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158751+0000", 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0750836960000001 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.158922+0000", 2024-09-16T11:16:36.987 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017088100000000001 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159141+0000", 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021923900000000001 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102742 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1085440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83466684999999, 2024-09-16T11:16:36.988 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.081381224, 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102742 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.989 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077921+0000", 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077922+0000", 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5199999999999998e-07 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077934+0000", 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2310000000000001e-05 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.990 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077937+0000", 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.63e-06 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077941+0000", 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.737e-06 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083705+0000", 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057636520000000002 2024-09-16T11:16:36.991 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083708+0000", 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3230000000000002e-06 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159176+0000", 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0754674479999999 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159225+0000", 2024-09-16T11:16:36.992 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9601999999999997e-05 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159302+0000", 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7269999999999997e-05 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102744 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1101824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83464303100001, 2024-09-16T11:16:36.993 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0817533610000001, 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102744 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077945+0000", 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8200000000000001e-07 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077947+0000", 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.939e-06 2024-09-16T11:16:36.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077953+0000", 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3079999999999998e-06 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077956+0000", 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.5659999999999998e-06 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.996 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083735+0000", 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00577831 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083738+0000", 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159355+0000", 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.075617115 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159488+0000", 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000132564 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159698+0000", 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00021088699999999999 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.998 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102746 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1118208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83461190899999, 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0818764009999999, 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102746 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:36.999 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077976+0000", 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077977+0000", 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4200000000000001e-07 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.000 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077979+0000", 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3769999999999999e-06 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077986+0000", 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9500000000000004e-06 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.001 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.077990+0000", 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.07e-06 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083763+0000", 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0057729260000000003 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083766+0000", 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3289999999999998e-06 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.002 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159725+0000", 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0759591660000001 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159771+0000", 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5068000000000003e-05 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159853+0000", 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2072999999999998e-05 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.003 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102747 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1134592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.83456416999999, 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0821684140000001, 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102747 2024-09-16T11:16:37.004 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.005 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078024+0000", 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9200000000000002e-07 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078026+0000", 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5260000000000001e-06 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078030+0000", 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.878e-06 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078032+0000", 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.34e-06 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083780+0000", 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.005748398 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.083784+0000", 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3390000000000001e-06 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.007 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.159898+0000", 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0761144309999999 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160040+0000", 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00014170799999999999 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160192+0000", 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000152502 2024-09-16T11:16:37.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:102749 4.d 4:b075a11b:::rbd_data.11b578dab209.0000000000000001:1b [read 1150976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 134.834542062, 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0827888530000001, 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.009 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 102749 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.010 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:22.078046+0000", 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093667+0000", 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0156210649999999 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093676+0000", 2024-09-16T11:16:37.011 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.1929999999999999e-06 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093707+0000", 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0817e-05 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.093721+0000", 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3427e-05 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.012 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160690+0000", 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.066969281000000005 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160768+0000", 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.8563e-05 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:23.160835+0000", 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6507000000000005e-05 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189141 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.425347697999996, 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2573314980000001, 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189141 2024-09-16T11:16:37.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487240+0000", 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487242+0000", 2024-09-16T11:16:37.015 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.776e-06 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487249+0000", 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3159999999999999e-06 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487266+0000", 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6282000000000001e-05 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.016 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487273+0000", 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4420000000000001e-06 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487360+0000", 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7185e-05 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487371+0000", 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1042e-05 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491386+0000", 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040141730000001 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491422+0000", 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6046e-05 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.738802+0000", 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2473805060000001 2024-09-16T11:16:37.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.739215+0000", 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041283699999999999 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.740124+0000", 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00090911699999999998 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.743920+0000", 2024-09-16T11:16:37.019 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.003796341 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744221+0000", 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00030062699999999998 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744339+0000", 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011752999999999999 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.020 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744572+0000", 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023327800000000001 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:189142 4.2 4:465839cb:::rbd_object_map.11b5e16f5aa1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 83.425301718, 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2577696710000001, 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:37.021 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 189142 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487286+0000", 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487287+0000", 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.1200000000000003e-07 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487290+0000", 2024-09-16T11:16:37.023 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7080000000000002e-06 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487299+0000", 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2439999999999999e-06 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487303+0000", 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9049999999999999e-06 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.024 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487415+0000", 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000112036 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:13.487422+0000", 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5379999999999997e-06 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491505+0000", 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0040826110000001 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:14.491528+0000", 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.2912999999999999e-05 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741313+0000", 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.249784893 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.741679+0000", 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00036654199999999997 2024-09-16T11:16:37.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 0", 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.742557+0000", 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00087796399999999996 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744195+0000", 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00163755 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:37.027 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744734+0000", 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00053935599999999995 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.744808+0000", 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4108000000000003e-05 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:15.745056+0000", 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00024779100000000003 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.029 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.029 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:37.030 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T11:16:37.134 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:37.134 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:37.135 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:37.135 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:37.144 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T11:16:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:37.246 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:37.246 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:37.246 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:37.256 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:828 4.7 e39/31 4:e2b8f5bc:::rbd_data.11b585795d6b.0000000000000002:head v 39'1, mlcod=0'0)", 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 196.48629431800001, 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.034406452999999997, 2024-09-16T11:16:37.352 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862954+0000", 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.862956+0000", 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.7630000000000001e-06 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877768+0000", 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.014812690999999999 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877809+0000", 2024-09-16T11:16:37.354 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.0932e-05 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877823+0000", 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4044e-05 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.877953+0000", 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00013006500000000001 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.355 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.878010+0000", 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7142999999999998e-05 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897316+0000", 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.019305480999999999 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:20.897360+0000", 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4333999999999999e-05 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.356 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48340 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.318322669, 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022726567999999999, 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48340 2024-09-16T11:16:37.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.358 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.359 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030925+0000", 2024-09-16T11:16:37.359 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030926+0000", 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0899999999999997e-07 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030928+0000", 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8640000000000001e-06 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030932+0000", 2024-09-16T11:16:37.360 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7099999999999998e-06 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030935+0000", 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.588e-06 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053558+0000", 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022623322000000001 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.361 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053603+0000", 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4320000000000003e-05 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053652+0000", 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9455e-05 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48341 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2007040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:37.362 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31831110799999, 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022841482, 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48341 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030937+0000", 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.364 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030939+0000", 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6330000000000001e-06 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030942+0000", 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4929999999999998e-06 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030946+0000", 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9029999999999997e-06 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.365 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053682+0000", 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022735828 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053730+0000", 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8205999999999999e-05 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.366 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053778+0000", 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.8143000000000003e-05 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48344 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2072576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.318297906, 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022921765, 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.367 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48344 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.368 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030950+0000", 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9999999999999999e-07 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030952+0000", 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.6309999999999999e-06 2024-09-16T11:16:37.369 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030956+0000", 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8099999999999999e-06 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030958+0000", 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.475e-06 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.370 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053793+0000", 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022834749000000001 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053833+0000", 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9644000000000002e-05 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053872+0000", 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9156000000000003e-05 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48376 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31828546599999, 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023071951, 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48376 2024-09-16T11:16:37.372 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.373 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030963+0000", 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8599999999999999e-07 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030964+0000", 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5370000000000001e-06 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030968+0000", 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.467e-06 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.374 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030971+0000", 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6040000000000001e-06 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053905+0000", 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022933937000000001 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.053958+0000", 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2083000000000003e-05 2024-09-16T11:16:37.375 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054035+0000", 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7037000000000001e-05 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48377 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1146880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.318271955, 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023192335000000001, 2024-09-16T11:16:37.376 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48377 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.377 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030976+0000", 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.9400000000000001e-07 2024-09-16T11:16:37.381 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030978+0000", 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.66e-06 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030982+0000", 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8249999999999998e-06 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.382 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030985+0000", 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8200000000000001e-06 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054050+0000", 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023065074000000001 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054094+0000", 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4323000000000002e-05 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.383 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054168+0000", 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4338999999999994e-05 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48378 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31826104000001, 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023385388999999999, 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.384 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48378 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:37.385 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030987+0000", 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7500000000000001e-07 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030989+0000", 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.57e-06 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.386 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030992+0000", 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.118e-06 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.030996+0000", 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.602e-06 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.387 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054216+0000", 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023220172000000001 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054301+0000", 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.4946999999999998e-05 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054372+0000", 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1705000000000005e-05 2024-09-16T11:16:37.388 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48379 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1277952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.318235122, 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023581504, 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.389 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48379 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.390 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031013+0000", 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6400000000000003e-07 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031016+0000", 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.103e-06 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031023+0000", 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.0779999999999998e-06 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.391 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031026+0000", 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.095e-06 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054402+0000", 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023375891999999999 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054500+0000", 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8876000000000007e-05 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054594+0000", 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.3995999999999999e-05 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48380 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1343488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31821966800001, 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023759203999999999, 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.393 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48380 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.394 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031028+0000", 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031029+0000", 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8500000000000002e-07 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031030+0000", 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5349999999999999e-06 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.395 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031034+0000", 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4489999999999999e-06 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031036+0000", 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.464e-06 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054638+0000", 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023602255999999999 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.396 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054718+0000", 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9523000000000004e-05 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054788+0000", 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9691999999999996e-05 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48381 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1409024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.397 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:37.398 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.318209782, 2024-09-16T11:16:37.398 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023969903000000001, 2024-09-16T11:16:37.398 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48381 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.399 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031038+0000", 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031039+0000", 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7099999999999998e-07 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031040+0000", 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3260000000000001e-06 2024-09-16T11:16:37.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031043+0000", 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.478e-06 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031046+0000", 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6280000000000001e-06 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054817+0000", 2024-09-16T11:16:37.401 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.023771502 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.054920+0000", 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000102415 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.402 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055008+0000", 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.8282999999999995e-05 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48382 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31820001899999, 2024-09-16T11:16:37.403 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024250675999999999, 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48382 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031048+0000", 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031050+0000", 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:37.405 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031053+0000", 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.309e-06 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031055+0000", 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3329999999999999e-06 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.406 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055067+0000", 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024011729999999998 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055192+0000", 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000124404 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055299+0000", 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000107216 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.407 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48383 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1540096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31818984200001, 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024444611000000002, 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48383 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.408 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031058+0000", 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031059+0000", 2024-09-16T11:16:37.409 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.34e-07 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031060+0000", 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.576e-06 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031065+0000", 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.2499999999999997e-06 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.410 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031069+0000", 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8e-06 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055348+0000", 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024278332 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055448+0000", 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000100358 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.411 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055503+0000", 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.4960999999999998e-05 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48384 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31817336, 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024653472999999999, 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.412 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48384 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031075+0000", 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2200000000000003e-07 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031077+0000", 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3480000000000001e-06 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031081+0000", 2024-09-16T11:16:37.414 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4580000000000003e-06 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031085+0000", 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2459999999999997e-06 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055555+0000", 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024469128999999999 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.415 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055642+0000", 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7247e-05 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055728+0000", 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6322999999999993e-05 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48385 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1671168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:37.416 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31815866299999, 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024788826999999999, 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48385 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.417 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031089+0000", 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031090+0000", 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8700000000000001e-07 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031092+0000", 2024-09-16T11:16:37.418 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4729999999999999e-06 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031099+0000", 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3269999999999997e-06 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031104+0000", 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3059999999999997e-06 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.419 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055773+0000", 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024669109000000002 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055830+0000", 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7398000000000002e-05 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055878+0000", 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7827000000000001e-05 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.420 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48386 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31814022, 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024947866999999999, 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48386 2024-09-16T11:16:37.421 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.422 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031108+0000", 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5299999999999999e-07 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031111+0000", 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.351e-06 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031117+0000", 2024-09-16T11:16:37.423 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5570000000000001e-06 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031121+0000", 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6789999999999998e-06 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055916+0000", 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024795228999999998 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.424 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.055989+0000", 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2849000000000007e-05 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056056+0000", 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6748999999999994e-05 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.425 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48387 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1802240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31812327099999, 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025055286999999999, 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48387 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:37.426 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031125+0000", 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2199999999999999e-07 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.427 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031127+0000", 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.091e-06 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031133+0000", 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9699999999999996e-06 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031137+0000", 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8589999999999998e-06 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.428 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056077+0000", 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.024939612999999999 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056132+0000", 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5415999999999999e-05 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056180+0000", 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7916e-05 2024-09-16T11:16:37.429 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48388 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31810405799999, 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025202362999999998, 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.430 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48388 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.431 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031144+0000", 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.3300000000000003e-07 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031147+0000", 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3319999999999999e-06 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031153+0000", 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3230000000000003e-06 2024-09-16T11:16:37.432 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031157+0000", 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9829999999999998e-06 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056213+0000", 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025056023 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.433 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056280+0000", 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6446999999999996e-05 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056346+0000", 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6822000000000005e-05 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48389 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:37.434 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31808684200001, 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025437115999999999, 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48389 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:37.435 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031161+0000", 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031162+0000", 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.01e-07 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.436 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031164+0000", 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.232e-06 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031170+0000", 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0610000000000001e-06 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031174+0000", 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6540000000000001e-06 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.437 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056384+0000", 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025210472000000001 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056518+0000", 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000133886 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056598+0000", 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.0409999999999998e-05 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.438 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48390 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.318065588, 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025637821000000002, 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48390 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.439 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031182+0000", 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031183+0000", 2024-09-16T11:16:37.440 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.9800000000000004e-07 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031186+0000", 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.5629999999999999e-06 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031192+0000", 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8430000000000004e-06 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031197+0000", 2024-09-16T11:16:37.441 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2930000000000002e-06 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056657+0000", 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025460011000000001 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056765+0000", 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000108497 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.442 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056820+0000", 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5115999999999999e-05 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:48391 4.8 4:1652c43b:::rbd_data.11b5eaebe1b2.0000000000000012:11 [read 2064384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.31804561999999, 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025833261, 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.443 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 48391 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031202+0000", 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.444 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031203+0000", 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1300000000000001e-07 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031205+0000", 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3369999999999998e-06 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031213+0000", 2024-09-16T11:16:37.445 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3649999999999998e-06 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.031217+0000", 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4179999999999999e-06 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056855+0000", 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.025638504999999999 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.056930+0000", 2024-09-16T11:16:37.446 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4109999999999993e-05 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.057036+0000", 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010611300000000001 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.447 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:37.448 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T11:16:37.480 INFO:teuthology.orchestra.run.smithi162.stderr:noscrub is unset 2024-09-16T11:16:37.495 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T11:16:37.553 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:37.553 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:37.553 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:37.553 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:37.561 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T11:16:37.657 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:37.658 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:37.658 INFO:teuthology.orchestra.run.smithi165.stdout: "complaint_time": 30, 2024-09-16T11:16:37.658 INFO:teuthology.orchestra.run.smithi165.stdout: "num_blocked_ops": 0 2024-09-16T11:16:37.658 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:37.666 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T11:16:37.768 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "size": 20, 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 600, 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [ 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50311 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.56827299, 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.12069964, 2024-09-16T11:16:37.769 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50311 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:37.770 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197704+0000", 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7599999999999998e-07 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.771 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197706+0000", 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.297e-06 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197709+0000", 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3409999999999999e-06 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197711+0000", 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3089999999999998e-06 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.772 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317800+0000", 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120088229 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.317948+0000", 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000148317 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318404+0000", 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000455871 2024-09-16T11:16:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50312 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.56825692300001, 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121329565, 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.774 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50312 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197720+0000", 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197721+0000", 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.27e-07 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197728+0000", 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4839999999999998e-06 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197732+0000", 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8120000000000001e-06 2024-09-16T11:16:37.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197734+0000", 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.6010000000000002e-06 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318482+0000", 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.120747971 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.318643+0000", 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160483 2024-09-16T11:16:37.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319050+0000", 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040688699999999998 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50313 4.1f 4:f9ba0657:::rbd_data.11b5eaebe1b2.0000000000000002:11 [read 1171456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.778 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.56824052799999, 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122000764, 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50313 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.779 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197737+0000", 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0699999999999998e-07 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197738+0000", 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.4079999999999999e-06 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197742+0000", 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.3129999999999999e-06 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197744+0000", 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3209999999999999e-06 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319094+0000", 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.121350031 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319245+0000", 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150715 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319737+0000", 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00049266900000000005 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50314 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.568230954, 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122131872, 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.783 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50314 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.784 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197746+0000", 2024-09-16T11:16:37.785 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.8299999999999998e-07 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197748+0000", 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1510000000000001e-06 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.788 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197751+0000", 2024-09-16T11:16:37.789 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.2899999999999998e-06 2024-09-16T11:16:37.789 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.789 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.789 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.789 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197753+0000", 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.419e-06 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319858+0000", 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122104859 2024-09-16T11:16:37.790 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319878+0000", 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9870000000000001e-05 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:50315 4.1f 4.ea605d9f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:37.791 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 169.56822141800001, 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.122193201, 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "reached pg", 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 50315 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:37.792 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.197756+0000", 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201713+0000", 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.0039577249999999996 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.793 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201719+0000", 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8860000000000001e-06 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201739+0000", 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9391000000000001e-05 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.201745+0000", 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5540000000000003e-06 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319912+0000", 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.118166967 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:13:48.319949+0000", 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.6677999999999997e-05 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.795 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_repop(client.4533.0:94659 4.e e76/31 4:76ab1a59:::rbd_data.11b56d1517d8.0000000000000002:head v 76'437, mlcod=74'435)", 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 140.870384535, 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.13823690299999999, 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.796 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895593+0000", 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.895595+0000", 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9630000000000001e-06 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917846+0000", 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.022251376999999999 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917880+0000", 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.4091999999999998e-05 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917890+0000", 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.6809999999999993e-06 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917952+0000", 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.2539999999999994e-05 2024-09-16T11:16:37.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:16.917997+0000", 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5173000000000003e-05 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033716+0000", 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.115718665 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.799 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:17.033830+0000", 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000113412 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:132661 4.5 4:a37e15d0:::rbd_data.11b5eb152d9.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 126.184549281, 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64773610599999998, 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:37.800 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 132661 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.801 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581428+0000", 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581429+0000", 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1540000000000001e-06 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581435+0000", 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.8250000000000001e-06 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.802 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581455+0000", 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0530000000000002e-05 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581463+0000", 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.1169999999999998e-06 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581526+0000", 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.3108999999999995e-05 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:31.581533+0000", 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.3150000000000003e-06 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.226963+0000", 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.64543014399999998 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227122+0000", 2024-09-16T11:16:37.804 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015865399999999999 2024-09-16T11:16:37.805 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.805 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 7", 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.227445+0000", 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032353699999999999 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228874+0000", 2024-09-16T11:16:37.806 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001428988 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228899+0000", 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4527000000000001e-05 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.228988+0000", 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9393000000000003e-05 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.807 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:14:32.229164+0000", 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017581300000000001 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185924 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:37.808 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.250029937999997, 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.931707379, 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185924 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.809 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515947+0000", 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.810 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515952+0000", 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.101e-06 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515962+0000", 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.7710000000000001e-06 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.515989+0000", 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7356e-05 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.811 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516001+0000", 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1506e-05 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516180+0000", 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000178779 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516204+0000", 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4428999999999999e-05 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.812 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447281+0000", 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93107732700000001 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447432+0000", 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000150901 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447655+0000", 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00022220900000000001 2024-09-16T11:16:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185930 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 708608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249949267000005, 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93195914300000005, 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.814 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185930 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516028+0000", 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.815 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516029+0000", 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.1230000000000001e-06 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516034+0000", 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.0780000000000003e-06 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516064+0000", 2024-09-16T11:16:37.816 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.0125e-05 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516073+0000", 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.7860000000000002e-06 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516304+0000", 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00023140500000000001 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.817 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516344+0000", 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9530999999999998e-05 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447692+0000", 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93134775299999994 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447796+0000", 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010442699999999999 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.818 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.447987+0000", 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000190915 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185934 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 704512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249827874999994, 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93209522300000003, 2024-09-16T11:16:37.819 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185934 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.820 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516149+0000", 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516150+0000", 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.074e-06 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516155+0000", 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.7589999999999997e-06 2024-09-16T11:16:37.821 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516183+0000", 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7820000000000001e-05 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516193+0000", 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.8779999999999993e-06 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516447+0000", 2024-09-16T11:16:37.822 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000254348 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516467+0000", 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9899e-05 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448012+0000", 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93154486700000005 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448089+0000", 2024-09-16T11:16:37.823 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.7521000000000005e-05 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448245+0000", 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000155057 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185935 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:37.824 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249775681000003, 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93237953399999995, 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185935 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.825 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516201+0000", 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516202+0000", 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.92e-07 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516205+0000", 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.123e-06 2024-09-16T11:16:37.826 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516212+0000", 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.7789999999999998e-06 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516217+0000", 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.6659999999999999e-06 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516537+0000", 2024-09-16T11:16:37.827 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00032066799999999997 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516545+0000", 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.4259999999999997e-06 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448282+0000", 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173715599999996 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.828 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448373+0000", 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.0511000000000001e-05 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448581+0000", 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00020851299999999999 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185956 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 681984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:37.829 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249718947999995, 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93265696300000001, 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185956 2024-09-16T11:16:37.830 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.831 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516258+0000", 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516259+0000", 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.8800000000000002e-07 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.832 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516263+0000", 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.9920000000000001e-06 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516273+0000", 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0033e-05 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516279+0000", 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5490000000000003e-06 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.833 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516627+0000", 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00034771900000000002 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516651+0000", 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.3725000000000001e-05 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448638+0000", 2024-09-16T11:16:37.834 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93198738800000003 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448734+0000", 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.5412000000000001e-05 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448915+0000", 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018145699999999999 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.835 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185961 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 684032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249671749000001, 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93293043399999998, 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185961 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.836 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516305+0000", 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516306+0000", 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.09e-07 2024-09-16T11:16:37.837 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516315+0000", 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.9770000000000006e-06 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516336+0000", 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0857e-05 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.838 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516346+0000", 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.0237e-05 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516705+0000", 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000358377 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516710+0000", 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.7780000000000004e-06 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.839 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.448968+0000", 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.932257683 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449068+0000", 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00010001300000000001 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449236+0000", 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000167703 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.840 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185966 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249621778999995, 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93317555699999999, 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.841 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185966 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516355+0000", 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516356+0000", 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9100000000000003e-07 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516360+0000", 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.2610000000000004e-06 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516373+0000", 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2388000000000001e-05 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.843 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516379+0000", 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0800000000000002e-06 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516790+0000", 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00041152700000000002 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516808+0000", 2024-09-16T11:16:37.844 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8054e-05 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449265+0000", 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93245662399999996 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449326+0000", 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.0933000000000001e-05 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.845 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449531+0000", 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000204899 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185967 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.249554665000005, 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93340519, 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.846 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185967 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.847 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516422+0000", 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516423+0000", 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.06e-07 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.848 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516427+0000", 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 3.8299999999999998e-06 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516435+0000", 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.354e-06 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516442+0000", 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.5579999999999997e-06 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.849 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516843+0000", 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00040154999999999998 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.516849+0000", 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.3669999999999996e-06 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449584+0000", 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93273523999999997 2024-09-16T11:16:37.850 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449667+0000", 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.2849000000000006e-05 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449828+0000", 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000160736 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185987 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 688128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.247579579000003, 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93173617600000003, 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185987 2024-09-16T11:16:37.852 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518398+0000", 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.853 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518399+0000", 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.387e-06 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518406+0000", 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.9659999999999999e-06 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518421+0000", 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5503999999999999e-05 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.854 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518429+0000", 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.2130000000000002e-06 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518583+0000", 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015405599999999999 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518607+0000", 2024-09-16T11:16:37.855 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.4192000000000001e-05 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449857+0000", 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93124967999999997 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.449969+0000", 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00011233100000000001 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.856 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450134+0000", 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000164847 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185988 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 694272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.247531093999996, 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93200475100000002, 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.857 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185988 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.858 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518446+0000", 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518447+0000", 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.9599999999999998e-07 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518451+0000", 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.4870000000000004e-06 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.859 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518472+0000", 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.0225e-05 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518478+0000", 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 6.6080000000000001e-06 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518697+0000", 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000219216 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518719+0000", 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1526999999999999e-05 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450171+0000", 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93145159399999999 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450282+0000", 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000111563 2024-09-16T11:16:37.861 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450451+0000", 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016873499999999999 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185993 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 690176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.247414066000005, 2024-09-16T11:16:37.862 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93220065500000004, 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185993 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.863 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518563+0000", 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518564+0000", 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.540000000000001e-07 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.864 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518570+0000", 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.5890000000000002e-06 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518589+0000", 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.9743000000000001e-05 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518598+0000", 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.6850000000000007e-06 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.865 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518785+0000", 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00018656900000000001 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518803+0000", 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.8813000000000001e-05 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450485+0000", 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93168132199999998 2024-09-16T11:16:37.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450591+0000", 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000106143 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.867 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450764+0000", 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00017283700000000001 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:185999 4.5 4:a4aa5056:::rbd_data.11b5e4cf0723.0000000000000009:28 [read 692224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:37.868 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 92.247322890999996, 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93242268299999997, 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "started", 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 185999 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.869 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518654+0000", 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518655+0000", 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 9.2600000000000001e-07 2024-09-16T11:16:37.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518660+0000", 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.5770000000000004e-06 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518673+0000", 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.3074000000000001e-05 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518695+0000", 2024-09-16T11:16:37.871 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.1739999999999999e-05 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518858+0000", 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00016340699999999999 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:05.518876+0000", 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.808e-05 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450799+0000", 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.93192298799999995 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.450915+0000", 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000115597 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:06.451077+0000", 2024-09-16T11:16:37.873 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.000162294 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "description": "osd_op(client.4533.0:196896 4.12 4:4c647cb5:::rbd_data.11b51fe98146.000000000000000f:head [delete] snapc 2b=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "initiated_at": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "age": 65.006916043999993, 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.88230661499999996, 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "type_data": { 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_info": { 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client": "client.4533", 2024-09-16T11:16:37.874 INFO:teuthology.orchestra.run.smithi165.stdout: "client_addr": "172.21.15.162:0/3651964385", 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "tid": 196896 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "events": [ 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "initiated", 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "header_read", 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759061+0000", 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.875 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "throttled", 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759063+0000", 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.5549999999999999e-06 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "all_read", 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759068+0000", 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.22e-06 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "dispatched", 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759096+0000", 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 2.7773e-05 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.876 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "queued_for_pg", 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759103+0000", 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 7.5399999999999998e-06 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759185+0000", 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1662000000000005e-05 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for peered", 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:32.759197+0000", 2024-09-16T11:16:37.877 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 1.2449000000000001e-05 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "reached_pg", 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637768+0000", 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.87857063999999996 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "started", 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.637922+0000", 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.00015380600000000001 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "waiting for subops from 1", 2024-09-16T11:16:37.878 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.638003+0000", 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 8.1586000000000001e-05 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "op_commit", 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.640115+0000", 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.002111351 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "sub_op_commit_rec", 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641268+0000", 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 0.001152817 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "commit_sent", 2024-09-16T11:16:37.879 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641327+0000", 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 5.9120999999999999e-05 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: "event": "done", 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: "time": "2024-09-16T11:15:33.641368+0000", 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: "duration": 4.1094999999999999e-05 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-09-16T11:16:37.880 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:38.489 INFO:teuthology.orchestra.run.smithi162.stderr:nodeep-scrub is unset 2024-09-16T11:16:39.388 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:16:39.383+0000 7fc3ad3e5540 -1 osd.7 181 log_to_monitors true 2024-09-16T11:16:39.497 INFO:tasks.ceph.osd.7.smithi165.stderr:2024-09-16T11:16:39.491+0000 7fc3a1928700 -1 osd.7 181 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:16:40.684 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T11:16:40.795 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-09-16T11:16:40.795 INFO:teuthology.orchestra.run.smithi165.stdout: "ops": [], 2024-09-16T11:16:40.796 INFO:teuthology.orchestra.run.smithi165.stdout: "num_ops": 0 2024-09-16T11:16:40.796 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-09-16T11:16:40.805 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-16T11:16:40.805 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-16T11:16:40.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-16T11:16:41.117 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:16:41.108+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e213 definitely_dead 0 2024-09-16T11:16:41.514 INFO:teuthology.orchestra.run.smithi162.stderr:marked in osd.7. 2024-09-16T11:16:41.526 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-16T11:16:41.526 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-16T11:16:41.527 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-16T11:16:41.527 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-16T11:16:41.826 INFO:tasks.ceph.mon.a.smithi162.stderr:2024-09-16T11:16:41.820+0000 7fcbd3f15700 -1 mon.a@0(leader).osd e214 definitely_dead 0 2024-09-16T11:16:42.524 INFO:teuthology.orchestra.run.smithi162.stderr:marked in osd.5. 2024-09-16T11:16:42.538 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-16T11:16:42.538 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T11:16:42.665 INFO:teuthology.orchestra.run.smithi162.stdout:mon.a: {} 2024-09-16T11:16:42.679 INFO:teuthology.orchestra.run.smithi162.stdout:mon.b: {} 2024-09-16T11:16:42.696 INFO:teuthology.orchestra.run.smithi162.stdout:mon.c: {} 2024-09-16T11:16:42.698 INFO:teuthology.orchestra.run.smithi162.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:16:42.698 INFO:teuthology.orchestra.run.smithi162.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:16:42.698 INFO:teuthology.orchestra.run.smithi162.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T11:16:42.711 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T11:16:43.531 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T11:16:43.542 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T11:16:44.523 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T11:16:44.535 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T11:16:45.478 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T11:16:45.489 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T11:16:46.484 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T11:16:46.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T11:16:47.488 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T11:16:47.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T11:16:48.483 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T11:16:48.497 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T11:16:49.567 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T11:16:49.578 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T11:16:50.480 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T11:16:50.491 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T11:16:51.482 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T11:16:51.495 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T11:16:52.601 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T11:16:52.614 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T11:16:53.604 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T11:16:53.619 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T11:16:54.623 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T11:16:54.637 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T11:16:55.628 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T11:16:55.642 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T11:16:56.639 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T11:16:56.653 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T11:16:57.654 INFO:teuthology.orchestra.run.smithi162.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T11:16:57.667 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T11:16:58.671 INFO:teuthology.orchestra.run.smithi162.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T11:16:58.687 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T11:16:58.688 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T11:16:58.688 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T11:16:58.688 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T11:16:58.688 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T11:16:58.688 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:16:58.982 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:16:58.982 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":231,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:16:58.650867+0000","last_up_change":"2024-09-16T11:16:40.476259+0000","last_in_change":"2024-09-16T11:16:42.499018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:12:53.361644+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:13:02.524586+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:13:11.876887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":87,"snap_epoch":206,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":181,"up_from":213,"up_thru":218,"down_at":182,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":32265},{"type":"v1","addr":"172.21.15.165:6820","nonce":32265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":32265},{"type":"v1","addr":"172.21.15.165:6825","nonce":32265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":32265},{"type":"v1","addr":"172.21.15.165:6831","nonce":32265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":32265},{"type":"v1","addr":"172.21.15.165:6829","nonce":32265}]},"public_addr":"172.21.15.165:6820/32265","cluster_addr":"172.21.15.165:6825/32265","heartbeat_back_addr":"172.21.15.165:6831/32265","heartbeat_front_addr":"172.21.15.165:6829/32265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":182}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:16:59.000 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T11:16:59.000 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:16:59.287 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:16:59.288 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":231,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:16:58.650867+0000","last_up_change":"2024-09-16T11:16:40.476259+0000","last_in_change":"2024-09-16T11:16:42.499018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:12:53.361644+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:13:02.524586+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:13:11.876887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":87,"snap_epoch":206,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":181,"up_from":213,"up_thru":218,"down_at":182,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":32265},{"type":"v1","addr":"172.21.15.165:6820","nonce":32265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":32265},{"type":"v1","addr":"172.21.15.165:6825","nonce":32265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":32265},{"type":"v1","addr":"172.21.15.165:6831","nonce":32265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":32265},{"type":"v1","addr":"172.21.15.165:6829","nonce":32265}]},"public_addr":"172.21.15.165:6820/32265","cluster_addr":"172.21.15.165:6825/32265","heartbeat_back_addr":"172.21.15.165:6831/32265","heartbeat_front_addr":"172.21.15.165:6829/32265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":182}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:16:59.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T11:16:59.302 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T11:16:59.303 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T11:16:59.303 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T11:16:59.304 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T11:16:59.304 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T11:16:59.305 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T11:16:59.305 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T11:16:59.508 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:16:59.508 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:16:59.510 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:16:59.510 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:16:59.514 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:16:59.514 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:16:59.515 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:16:59.516 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:16:59.526 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:16:59.526 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:16:59.532 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:16:59.532 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:16:59.533 INFO:teuthology.orchestra.run.smithi162.stdout:914828034054 2024-09-16T11:16:59.534 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:16:59.542 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:16:59.543 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:16:59.950 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:16:59.957 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:16:59.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.1 2024-09-16T11:16:59.974 INFO:teuthology.orchestra.run.smithi162.stdout:55834574901 2024-09-16T11:16:59.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.3 2024-09-16T11:16:59.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574902 got 55834574901 for osd.6 2024-09-16T11:16:59.997 INFO:teuthology.orchestra.run.smithi162.stdout:55834574901 2024-09-16T11:16:59.997 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:17:00.009 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:17:00.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574902 got 55834574901 for osd.0 2024-09-16T11:17:00.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.2 2024-09-16T11:17:00.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.4 2024-09-16T11:17:00.028 INFO:teuthology.orchestra.run.smithi162.stdout:914828034053 2024-09-16T11:17:00.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 914828034054 got 914828034053 for osd.7 2024-09-16T11:17:00.047 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:17:00.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.5 2024-09-16T11:17:00.974 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:17:00.982 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T11:17:00.992 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T11:17:01.016 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:17:01.017 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:17:01.023 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T11:17:01.040 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T11:17:01.060 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T11:17:01.426 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:17:01.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.1 2024-09-16T11:17:01.442 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.446 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:17:01.455 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:17:01.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.3 2024-09-16T11:17:01.464 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574902 got 55834574902 for osd.6 2024-09-16T11:17:01.468 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.479 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:17:01.484 INFO:teuthology.orchestra.run.smithi162.stdout:914828034054 2024-09-16T11:17:01.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 914828034054 got 914828034054 for osd.7 2024-09-16T11:17:01.498 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.4 2024-09-16T11:17:01.498 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.507 INFO:teuthology.orchestra.run.smithi162.stdout:55834574902 2024-09-16T11:17:01.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574902 got 55834574902 for osd.0 2024-09-16T11:17:01.518 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.527 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:17:01.532 INFO:teuthology.orchestra.run.smithi162.stdout:55834574903 2024-09-16T11:17:01.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.5 2024-09-16T11:17:01.541 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.2 2024-09-16T11:17:01.548 DEBUG:teuthology.parallel:result is None 2024-09-16T11:17:01.548 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T11:17:01.548 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:01.823 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:01.826 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:01.836 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-16T11:17:00.013722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23554,"ondisk_log_size":23554,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444152,"kb_used_data":23028,"kb_used_omap":48,"kb_used_meta":2420943,"kb_avail":752530568,"statfs":{"total":773094113280,"available":770591301632,"internally_reserved":0,"allocated":23580672,"data_stored":12634200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2479045869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.401737"},"pg_stats":[{"pgid":"4.1f","version":"206'498","reported_seq":7631,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969898+0000","last_change":"2024-09-16T11:16:46.480011+0000","last_active":"2024-09-16T11:16:57.969898+0000","last_peered":"2024-09-16T11:16:57.969898+0000","last_clean":"2024-09-16T11:16:57.969898+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:16:57.969898+0000","last_undegraded":"2024-09-16T11:16:57.969898+0000","last_fullsized":"2024-09-16T11:16:57.969898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:17.579128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12720,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644207+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:57.644207+0000","last_peered":"2024-09-16T11:16:57.644207+0000","last_clean":"2024-09-16T11:16:57.644207+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:57.644207+0000","last_undegraded":"2024-09-16T11:16:57.644207+0000","last_fullsized":"2024-09-16T11:16:57.644207+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:05.159446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"205'510","reported_seq":5433,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970493+0000","last_change":"2024-09-16T11:16:34.642934+0000","last_active":"2024-09-16T11:16:57.970493+0000","last_peered":"2024-09-16T11:16:57.970493+0000","last_clean":"2024-09-16T11:16:57.970493+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:16:57.970493+0000","last_undegraded":"2024-09-16T11:16:57.970493+0000","last_fullsized":"2024-09-16T11:16:57.970493+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:00.434118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"206'1074","reported_seq":9478,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969101+0000","last_change":"2024-09-16T11:16:38.490863+0000","last_active":"2024-09-16T11:16:57.969101+0000","last_peered":"2024-09-16T11:16:57.969101+0000","last_clean":"2024-09-16T11:16:57.969101+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:16:57.969101+0000","last_undegraded":"2024-09-16T11:16:57.969101+0000","last_fullsized":"2024-09-16T11:16:57.969101+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:07.899214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7521,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646819+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:57.646819+0000","last_peered":"2024-09-16T11:16:57.646819+0000","last_clean":"2024-09-16T11:16:57.646819+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:57.646819+0000","last_undegraded":"2024-09-16T11:16:57.646819+0000","last_fullsized":"2024-09-16T11:16:57.646819+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"206'701","reported_seq":8134,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646713+0000","last_change":"2024-09-16T11:16:33.634941+0000","last_active":"2024-09-16T11:16:57.646713+0000","last_peered":"2024-09-16T11:16:57.646713+0000","last_clean":"2024-09-16T11:16:57.646713+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:16:57.646713+0000","last_undegraded":"2024-09-16T11:16:57.646713+0000","last_fullsized":"2024-09-16T11:16:57.646713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"199'840","reported_seq":3434,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970116+0000","last_change":"2024-09-16T11:16:33.627164+0000","last_active":"2024-09-16T11:16:57.970116+0000","last_peered":"2024-09-16T11:16:57.970116+0000","last_clean":"2024-09-16T11:16:57.970116+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:16:57.970116+0000","last_undegraded":"2024-09-16T11:16:57.970116+0000","last_fullsized":"2024-09-16T11:16:57.970116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"206'775","reported_seq":5543,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646596+0000","last_change":"2024-09-16T11:16:33.633666+0000","last_active":"2024-09-16T11:16:57.646596+0000","last_peered":"2024-09-16T11:16:57.646596+0000","last_clean":"2024-09-16T11:16:57.646596+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:16:57.646596+0000","last_undegraded":"2024-09-16T11:16:57.646596+0000","last_fullsized":"2024-09-16T11:16:57.646596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7723,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969928+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:57.969928+0000","last_peered":"2024-09-16T11:16:57.969928+0000","last_clean":"2024-09-16T11:16:57.969928+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:57.969928+0000","last_undegraded":"2024-09-16T11:16:57.969928+0000","last_fullsized":"2024-09-16T11:16:57.969928+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:42.706001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"200'499","reported_seq":7620,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644530+0000","last_change":"2024-09-16T11:16:33.624591+0000","last_active":"2024-09-16T11:16:57.644530+0000","last_peered":"2024-09-16T11:16:57.644530+0000","last_clean":"2024-09-16T11:16:57.644530+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:16:57.644530+0000","last_undegraded":"2024-09-16T11:16:57.644530+0000","last_fullsized":"2024-09-16T11:16:57.644530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"205'486","reported_seq":4883,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648892+0000","last_change":"2024-09-16T11:16:45.480011+0000","last_active":"2024-09-16T11:16:57.648892+0000","last_peered":"2024-09-16T11:16:57.648892+0000","last_clean":"2024-09-16T11:16:57.648892+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:16:57.648892+0000","last_undegraded":"2024-09-16T11:16:57.648892+0000","last_fullsized":"2024-09-16T11:16:57.648892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:23.227245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"205'673","reported_seq":7416,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969848+0000","last_change":"2024-09-16T11:16:39.492405+0000","last_active":"2024-09-16T11:16:57.969848+0000","last_peered":"2024-09-16T11:16:57.969848+0000","last_clean":"2024-09-16T11:16:57.969848+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:16:57.969848+0000","last_undegraded":"2024-09-16T11:16:57.969848+0000","last_fullsized":"2024-09-16T11:16:57.969848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:21.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"205'593","reported_seq":4530,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647123+0000","last_change":"2024-09-16T11:16:33.633503+0000","last_active":"2024-09-16T11:16:57.647123+0000","last_peered":"2024-09-16T11:16:57.647123+0000","last_clean":"2024-09-16T11:16:57.647123+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:16:57.647123+0000","last_undegraded":"2024-09-16T11:16:57.647123+0000","last_fullsized":"2024-09-16T11:16:57.647123+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"206'531","reported_seq":6108,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968113+0000","last_change":"2024-09-16T11:16:47.482430+0000","last_active":"2024-09-16T11:16:57.968113+0000","last_peered":"2024-09-16T11:16:57.968113+0000","last_clean":"2024-09-16T11:16:57.968113+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:16:57.968113+0000","last_undegraded":"2024-09-16T11:16:57.968113+0000","last_fullsized":"2024-09-16T11:16:57.968113+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:41.497485+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8207,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969889+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:57.969889+0000","last_peered":"2024-09-16T11:16:57.969889+0000","last_clean":"2024-09-16T11:16:57.969889+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:57.969889+0000","last_undegraded":"2024-09-16T11:16:57.969889+0000","last_fullsized":"2024-09-16T11:16:57.969889+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:05.400722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"193'627","reported_seq":7679,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969692+0000","last_change":"2024-09-16T11:16:35.483110+0000","last_active":"2024-09-16T11:16:57.969692+0000","last_peered":"2024-09-16T11:16:57.969692+0000","last_clean":"2024-09-16T11:16:57.969692+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:16:57.969692+0000","last_undegraded":"2024-09-16T11:16:57.969692+0000","last_fullsized":"2024-09-16T11:16:57.969692+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:27.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969871+0000","last_change":"2024-09-16T11:16:42.522836+0000","last_active":"2024-09-16T11:16:57.969871+0000","last_peered":"2024-09-16T11:16:57.969871+0000","last_clean":"2024-09-16T11:16:57.969871+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:16:57.969871+0000","last_undegraded":"2024-09-16T11:16:57.969871+0000","last_fullsized":"2024-09-16T11:16:57.969871+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:31.057363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"200'1007","reported_seq":6373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646875+0000","last_change":"2024-09-16T11:16:46.477020+0000","last_active":"2024-09-16T11:16:57.646875+0000","last_peered":"2024-09-16T11:16:57.646875+0000","last_clean":"2024-09-16T11:16:57.646875+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:16:57.646875+0000","last_undegraded":"2024-09-16T11:16:57.646875+0000","last_fullsized":"2024-09-16T11:16:57.646875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:46.468919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968883+0000","last_change":"2024-09-16T11:16:47.479923+0000","last_active":"2024-09-16T11:16:57.968883+0000","last_peered":"2024-09-16T11:16:57.968883+0000","last_clean":"2024-09-16T11:16:57.968883+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:16:57.968883+0000","last_undegraded":"2024-09-16T11:16:57.968883+0000","last_fullsized":"2024-09-16T11:16:57.968883+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:03.586218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6183,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970015+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:57.970015+0000","last_peered":"2024-09-16T11:16:57.970015+0000","last_clean":"2024-09-16T11:16:57.970015+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:57.970015+0000","last_undegraded":"2024-09-16T11:16:57.970015+0000","last_fullsized":"2024-09-16T11:16:57.970015+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:31.708609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969274+0000","last_change":"2024-09-16T11:16:42.520140+0000","last_active":"2024-09-16T11:16:57.969274+0000","last_peered":"2024-09-16T11:16:57.969274+0000","last_clean":"2024-09-16T11:16:57.969274+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:16:57.969274+0000","last_undegraded":"2024-09-16T11:16:57.969274+0000","last_fullsized":"2024-09-16T11:16:57.969274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:01.926666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"207'1204","reported_seq":18994,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644171+0000","last_change":"2024-09-16T11:16:47.485172+0000","last_active":"2024-09-16T11:16:57.644171+0000","last_peered":"2024-09-16T11:16:57.644171+0000","last_clean":"2024-09-16T11:16:57.644171+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:16:57.644171+0000","last_undegraded":"2024-09-16T11:16:57.644171+0000","last_fullsized":"2024-09-16T11:16:57.644171+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:10.056185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968679+0000","last_change":"2024-09-16T11:16:47.480778+0000","last_active":"2024-09-16T11:16:57.968679+0000","last_peered":"2024-09-16T11:16:57.968679+0000","last_clean":"2024-09-16T11:16:57.968679+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:16:57.968679+0000","last_undegraded":"2024-09-16T11:16:57.968679+0000","last_fullsized":"2024-09-16T11:16:57.968679+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:29.212441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"205'491","reported_seq":8704,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968268+0000","last_change":"2024-09-16T11:16:47.486651+0000","last_active":"2024-09-16T11:16:57.968268+0000","last_peered":"2024-09-16T11:16:57.968268+0000","last_clean":"2024-09-16T11:16:57.968268+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:16:57.968268+0000","last_undegraded":"2024-09-16T11:16:57.968268+0000","last_fullsized":"2024-09-16T11:16:57.968268+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:45.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647126+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:16:57.647126+0000","last_peered":"2024-09-16T11:16:57.647126+0000","last_clean":"2024-09-16T11:16:57.647126+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:16:57.647126+0000","last_undegraded":"2024-09-16T11:16:57.647126+0000","last_fullsized":"2024-09-16T11:16:57.647126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648781+0000","last_change":"2024-09-16T11:16:43.567954+0000","last_active":"2024-09-16T11:16:57.648781+0000","last_peered":"2024-09-16T11:16:57.648781+0000","last_clean":"2024-09-16T11:16:57.648781+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:16:57.648781+0000","last_undegraded":"2024-09-16T11:16:57.648781+0000","last_fullsized":"2024-09-16T11:16:57.648781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:22:11.421317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"206'644","reported_seq":6808,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968521+0000","last_change":"2024-09-16T11:16:47.483421+0000","last_active":"2024-09-16T11:16:57.968521+0000","last_peered":"2024-09-16T11:16:57.968521+0000","last_clean":"2024-09-16T11:16:57.968521+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:16:57.968521+0000","last_undegraded":"2024-09-16T11:16:57.968521+0000","last_fullsized":"2024-09-16T11:16:57.968521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:13.778395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":168,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.967896+0000","last_change":"2024-09-16T11:16:47.481364+0000","last_active":"2024-09-16T11:16:57.967896+0000","last_peered":"2024-09-16T11:16:57.967896+0000","last_clean":"2024-09-16T11:16:57.967896+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:16:57.967896+0000","last_undegraded":"2024-09-16T11:16:57.967896+0000","last_fullsized":"2024-09-16T11:16:57.967896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:22.268248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969784+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:16:57.969784+0000","last_peered":"2024-09-16T11:16:57.969784+0000","last_clean":"2024-09-16T11:16:57.969784+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:16:57.969784+0000","last_undegraded":"2024-09-16T11:16:57.969784+0000","last_fullsized":"2024-09-16T11:16:57.969784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"193'684","reported_seq":5002,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969111+0000","last_change":"2024-09-16T11:16:50.477765+0000","last_active":"2024-09-16T11:16:57.969111+0000","last_peered":"2024-09-16T11:16:57.969111+0000","last_clean":"2024-09-16T11:16:57.969111+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:16:57.969111+0000","last_undegraded":"2024-09-16T11:16:57.969111+0000","last_fullsized":"2024-09-16T11:16:57.969111+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:14.471034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"212'2","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644047+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:16:57.644047+0000","last_peered":"2024-09-16T11:16:57.644047+0000","last_clean":"2024-09-16T11:16:57.644047+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:16:57.644047+0000","last_undegraded":"2024-09-16T11:16:57.644047+0000","last_fullsized":"2024-09-16T11:16:57.644047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969075+0000","last_change":"2024-09-16T11:16:42.521626+0000","last_active":"2024-09-16T11:16:57.969075+0000","last_peered":"2024-09-16T11:16:57.969075+0000","last_clean":"2024-09-16T11:16:57.969075+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:16:57.969075+0000","last_undegraded":"2024-09-16T11:16:57.969075+0000","last_fullsized":"2024-09-16T11:16:57.969075+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:03.898472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"207'538","reported_seq":4282,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644466+0000","last_change":"2024-09-16T11:16:36.479810+0000","last_active":"2024-09-16T11:16:57.644466+0000","last_peered":"2024-09-16T11:16:57.644466+0000","last_clean":"2024-09-16T11:16:57.644466+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:16:57.644466+0000","last_undegraded":"2024-09-16T11:16:57.644466+0000","last_fullsized":"2024-09-16T11:16:57.644466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:09.533692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"168'4","reported_seq":289,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969684+0000","last_change":"2024-09-16T11:16:43.545938+0000","last_active":"2024-09-16T11:16:57.969684+0000","last_peered":"2024-09-16T11:16:57.969684+0000","last_clean":"2024-09-16T11:16:57.969684+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:16:57.969684+0000","last_undegraded":"2024-09-16T11:16:57.969684+0000","last_fullsized":"2024-09-16T11:16:57.969684+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:35.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648992+0000","last_change":"2024-09-16T11:16:43.565231+0000","last_active":"2024-09-16T11:16:57.648992+0000","last_peered":"2024-09-16T11:16:57.648992+0000","last_clean":"2024-09-16T11:16:57.648992+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:16:57.648992+0000","last_undegraded":"2024-09-16T11:16:57.648992+0000","last_fullsized":"2024-09-16T11:16:57.648992+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:45.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":30,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970000+0000","last_change":"2024-09-16T11:16:42.529324+0000","last_active":"2024-09-16T11:16:57.970000+0000","last_peered":"2024-09-16T11:16:57.970000+0000","last_clean":"2024-09-16T11:16:57.970000+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:16:57.970000+0000","last_undegraded":"2024-09-16T11:16:57.970000+0000","last_fullsized":"2024-09-16T11:16:57.970000+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:09.000275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"205'573","reported_seq":6327,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:33.633708+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"202'885","reported_seq":3505,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648499+0000","last_change":"2024-09-16T11:16:48.483747+0000","last_active":"2024-09-16T11:16:57.648499+0000","last_peered":"2024-09-16T11:16:57.648499+0000","last_clean":"2024-09-16T11:16:57.648499+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:16:57.648499+0000","last_undegraded":"2024-09-16T11:16:57.648499+0000","last_fullsized":"2024-09-16T11:16:57.648499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11038,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646703+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:57.646703+0000","last_peered":"2024-09-16T11:16:57.646703+0000","last_clean":"2024-09-16T11:16:57.646703+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:57.646703+0000","last_undegraded":"2024-09-16T11:16:57.646703+0000","last_fullsized":"2024-09-16T11:16:57.646703+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:08.661865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"205'573","reported_seq":5145,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:37.481009+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:45.260823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"212'668","reported_seq":8071,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644042+0000","last_change":"2024-09-16T11:16:39.501093+0000","last_active":"2024-09-16T11:16:57.644042+0000","last_peered":"2024-09-16T11:16:57.644042+0000","last_clean":"2024-09-16T11:16:57.644042+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:16:57.644042+0000","last_undegraded":"2024-09-16T11:16:57.644042+0000","last_fullsized":"2024-09-16T11:16:57.644042+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:46.228140+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"206'1205","reported_seq":10338,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646826+0000","last_change":"2024-09-16T11:16:33.632197+0000","last_active":"2024-09-16T11:16:57.646826+0000","last_peered":"2024-09-16T11:16:57.646826+0000","last_clean":"2024-09-16T11:16:57.646826+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:16:57.646826+0000","last_undegraded":"2024-09-16T11:16:57.646826+0000","last_fullsized":"2024-09-16T11:16:57.646826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17294,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647063+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:57.647063+0000","last_peered":"2024-09-16T11:16:57.647063+0000","last_clean":"2024-09-16T11:16:57.647063+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:57.647063+0000","last_undegraded":"2024-09-16T11:16:57.647063+0000","last_fullsized":"2024-09-16T11:16:57.647063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"200'769","reported_seq":8815,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647188+0000","last_change":"2024-09-16T11:16:33.635099+0000","last_active":"2024-09-16T11:16:57.647188+0000","last_peered":"2024-09-16T11:16:57.647188+0000","last_clean":"2024-09-16T11:16:57.647188+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:16:57.647188+0000","last_undegraded":"2024-09-16T11:16:57.647188+0000","last_fullsized":"2024-09-16T11:16:57.647188+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"200'540","reported_seq":5263,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969308+0000","last_change":"2024-09-16T11:16:47.488023+0000","last_active":"2024-09-16T11:16:57.969308+0000","last_peered":"2024-09-16T11:16:57.969308+0000","last_clean":"2024-09-16T11:16:57.969308+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:16:57.969308+0000","last_undegraded":"2024-09-16T11:16:57.969308+0000","last_fullsized":"2024-09-16T11:16:57.969308+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:54.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23473,"ondisk_log_size":23473,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034054,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305468,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066372,"statfs":{"total":96636764160,"available":96323964928,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300092,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071748,"statfs":{"total":96636764160,"available":96329469952,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.0600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":1.296},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.999}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":1.1060000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.2390000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.403},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.462},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.879}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.085},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.157}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.002}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.052},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.286}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312580,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059260,"statfs":{"total":96636764160,"available":96316682240,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.139,"5min":1.329,"15min":1.329},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1379999999999999,"5min":1.173,"15min":1.173},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86699999999999999,"15min":0.90100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.874,"15min":0.91800000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.026,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0029999999999999,"15min":1.0469999999999999},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.304,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.278}]},{"osd":3,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0649999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.58299999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1120000000000001,"5min":2.077,"15min":2.077},"last":1.3129999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2829999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.417},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0980000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.069,"15min":1.1299999999999999},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.256,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.119,"15min":1.1419999999999999},"min":{"1min":0.627,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.355,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.508}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.456},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.6399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.503},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2785280,"data_stored":1427036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":2.0720000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":2.1520000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.498},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308412,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063428,"statfs":{"total":96636764160,"available":96320950272,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":1.0980000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.212},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":1.371}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":1.0029999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.4530000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":1.4039999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313148,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058692,"statfs":{"total":96636764160,"available":96316100608,"internally_reserved":0,"allocated":3383296,"data_stored":2017487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.036}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.27},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":1.3160000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.22},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.8400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:01.839 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:02.118 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:02.121 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:02.130 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T11:17:02.014261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23554,"ondisk_log_size":23554,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444152,"kb_used_data":23028,"kb_used_omap":48,"kb_used_meta":2420943,"kb_avail":752530568,"statfs":{"total":773094113280,"available":770591301632,"internally_reserved":0,"allocated":23580672,"data_stored":12634200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2479045869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001566"},"pg_stats":[{"pgid":"4.1f","version":"206'498","reported_seq":7631,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969898+0000","last_change":"2024-09-16T11:16:46.480011+0000","last_active":"2024-09-16T11:16:57.969898+0000","last_peered":"2024-09-16T11:16:57.969898+0000","last_clean":"2024-09-16T11:16:57.969898+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:16:57.969898+0000","last_undegraded":"2024-09-16T11:16:57.969898+0000","last_fullsized":"2024-09-16T11:16:57.969898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:17.579128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12720,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644207+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:57.644207+0000","last_peered":"2024-09-16T11:16:57.644207+0000","last_clean":"2024-09-16T11:16:57.644207+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:57.644207+0000","last_undegraded":"2024-09-16T11:16:57.644207+0000","last_fullsized":"2024-09-16T11:16:57.644207+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:05.159446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"205'510","reported_seq":5433,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970493+0000","last_change":"2024-09-16T11:16:34.642934+0000","last_active":"2024-09-16T11:16:57.970493+0000","last_peered":"2024-09-16T11:16:57.970493+0000","last_clean":"2024-09-16T11:16:57.970493+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:16:57.970493+0000","last_undegraded":"2024-09-16T11:16:57.970493+0000","last_fullsized":"2024-09-16T11:16:57.970493+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:00.434118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"206'1074","reported_seq":9478,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969101+0000","last_change":"2024-09-16T11:16:38.490863+0000","last_active":"2024-09-16T11:16:57.969101+0000","last_peered":"2024-09-16T11:16:57.969101+0000","last_clean":"2024-09-16T11:16:57.969101+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:16:57.969101+0000","last_undegraded":"2024-09-16T11:16:57.969101+0000","last_fullsized":"2024-09-16T11:16:57.969101+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:07.899214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7521,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646819+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:57.646819+0000","last_peered":"2024-09-16T11:16:57.646819+0000","last_clean":"2024-09-16T11:16:57.646819+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:57.646819+0000","last_undegraded":"2024-09-16T11:16:57.646819+0000","last_fullsized":"2024-09-16T11:16:57.646819+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"206'701","reported_seq":8134,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646713+0000","last_change":"2024-09-16T11:16:33.634941+0000","last_active":"2024-09-16T11:16:57.646713+0000","last_peered":"2024-09-16T11:16:57.646713+0000","last_clean":"2024-09-16T11:16:57.646713+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:16:57.646713+0000","last_undegraded":"2024-09-16T11:16:57.646713+0000","last_fullsized":"2024-09-16T11:16:57.646713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"199'840","reported_seq":3434,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970116+0000","last_change":"2024-09-16T11:16:33.627164+0000","last_active":"2024-09-16T11:16:57.970116+0000","last_peered":"2024-09-16T11:16:57.970116+0000","last_clean":"2024-09-16T11:16:57.970116+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:16:57.970116+0000","last_undegraded":"2024-09-16T11:16:57.970116+0000","last_fullsized":"2024-09-16T11:16:57.970116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"206'775","reported_seq":5543,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646596+0000","last_change":"2024-09-16T11:16:33.633666+0000","last_active":"2024-09-16T11:16:57.646596+0000","last_peered":"2024-09-16T11:16:57.646596+0000","last_clean":"2024-09-16T11:16:57.646596+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:16:57.646596+0000","last_undegraded":"2024-09-16T11:16:57.646596+0000","last_fullsized":"2024-09-16T11:16:57.646596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7723,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969928+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:57.969928+0000","last_peered":"2024-09-16T11:16:57.969928+0000","last_clean":"2024-09-16T11:16:57.969928+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:57.969928+0000","last_undegraded":"2024-09-16T11:16:57.969928+0000","last_fullsized":"2024-09-16T11:16:57.969928+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:42.706001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"200'499","reported_seq":7620,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644530+0000","last_change":"2024-09-16T11:16:33.624591+0000","last_active":"2024-09-16T11:16:57.644530+0000","last_peered":"2024-09-16T11:16:57.644530+0000","last_clean":"2024-09-16T11:16:57.644530+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:16:57.644530+0000","last_undegraded":"2024-09-16T11:16:57.644530+0000","last_fullsized":"2024-09-16T11:16:57.644530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"205'486","reported_seq":4883,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648892+0000","last_change":"2024-09-16T11:16:45.480011+0000","last_active":"2024-09-16T11:16:57.648892+0000","last_peered":"2024-09-16T11:16:57.648892+0000","last_clean":"2024-09-16T11:16:57.648892+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:16:57.648892+0000","last_undegraded":"2024-09-16T11:16:57.648892+0000","last_fullsized":"2024-09-16T11:16:57.648892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:23.227245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"205'673","reported_seq":7416,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969848+0000","last_change":"2024-09-16T11:16:39.492405+0000","last_active":"2024-09-16T11:16:57.969848+0000","last_peered":"2024-09-16T11:16:57.969848+0000","last_clean":"2024-09-16T11:16:57.969848+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:16:57.969848+0000","last_undegraded":"2024-09-16T11:16:57.969848+0000","last_fullsized":"2024-09-16T11:16:57.969848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:21.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"205'593","reported_seq":4530,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647123+0000","last_change":"2024-09-16T11:16:33.633503+0000","last_active":"2024-09-16T11:16:57.647123+0000","last_peered":"2024-09-16T11:16:57.647123+0000","last_clean":"2024-09-16T11:16:57.647123+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:16:57.647123+0000","last_undegraded":"2024-09-16T11:16:57.647123+0000","last_fullsized":"2024-09-16T11:16:57.647123+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"206'531","reported_seq":6108,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968113+0000","last_change":"2024-09-16T11:16:47.482430+0000","last_active":"2024-09-16T11:16:57.968113+0000","last_peered":"2024-09-16T11:16:57.968113+0000","last_clean":"2024-09-16T11:16:57.968113+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:16:57.968113+0000","last_undegraded":"2024-09-16T11:16:57.968113+0000","last_fullsized":"2024-09-16T11:16:57.968113+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:41.497485+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8207,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969889+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:57.969889+0000","last_peered":"2024-09-16T11:16:57.969889+0000","last_clean":"2024-09-16T11:16:57.969889+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:57.969889+0000","last_undegraded":"2024-09-16T11:16:57.969889+0000","last_fullsized":"2024-09-16T11:16:57.969889+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:05.400722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"193'627","reported_seq":7679,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969692+0000","last_change":"2024-09-16T11:16:35.483110+0000","last_active":"2024-09-16T11:16:57.969692+0000","last_peered":"2024-09-16T11:16:57.969692+0000","last_clean":"2024-09-16T11:16:57.969692+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:16:57.969692+0000","last_undegraded":"2024-09-16T11:16:57.969692+0000","last_fullsized":"2024-09-16T11:16:57.969692+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:27.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969871+0000","last_change":"2024-09-16T11:16:42.522836+0000","last_active":"2024-09-16T11:16:57.969871+0000","last_peered":"2024-09-16T11:16:57.969871+0000","last_clean":"2024-09-16T11:16:57.969871+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:16:57.969871+0000","last_undegraded":"2024-09-16T11:16:57.969871+0000","last_fullsized":"2024-09-16T11:16:57.969871+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:31.057363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"200'1007","reported_seq":6373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646875+0000","last_change":"2024-09-16T11:16:46.477020+0000","last_active":"2024-09-16T11:16:57.646875+0000","last_peered":"2024-09-16T11:16:57.646875+0000","last_clean":"2024-09-16T11:16:57.646875+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:16:57.646875+0000","last_undegraded":"2024-09-16T11:16:57.646875+0000","last_fullsized":"2024-09-16T11:16:57.646875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:46.468919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968883+0000","last_change":"2024-09-16T11:16:47.479923+0000","last_active":"2024-09-16T11:16:57.968883+0000","last_peered":"2024-09-16T11:16:57.968883+0000","last_clean":"2024-09-16T11:16:57.968883+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:16:57.968883+0000","last_undegraded":"2024-09-16T11:16:57.968883+0000","last_fullsized":"2024-09-16T11:16:57.968883+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:03.586218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6183,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970015+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:57.970015+0000","last_peered":"2024-09-16T11:16:57.970015+0000","last_clean":"2024-09-16T11:16:57.970015+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:57.970015+0000","last_undegraded":"2024-09-16T11:16:57.970015+0000","last_fullsized":"2024-09-16T11:16:57.970015+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:31.708609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969274+0000","last_change":"2024-09-16T11:16:42.520140+0000","last_active":"2024-09-16T11:16:57.969274+0000","last_peered":"2024-09-16T11:16:57.969274+0000","last_clean":"2024-09-16T11:16:57.969274+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:16:57.969274+0000","last_undegraded":"2024-09-16T11:16:57.969274+0000","last_fullsized":"2024-09-16T11:16:57.969274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:01.926666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"207'1204","reported_seq":18994,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644171+0000","last_change":"2024-09-16T11:16:47.485172+0000","last_active":"2024-09-16T11:16:57.644171+0000","last_peered":"2024-09-16T11:16:57.644171+0000","last_clean":"2024-09-16T11:16:57.644171+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:16:57.644171+0000","last_undegraded":"2024-09-16T11:16:57.644171+0000","last_fullsized":"2024-09-16T11:16:57.644171+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:10.056185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968679+0000","last_change":"2024-09-16T11:16:47.480778+0000","last_active":"2024-09-16T11:16:57.968679+0000","last_peered":"2024-09-16T11:16:57.968679+0000","last_clean":"2024-09-16T11:16:57.968679+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:16:57.968679+0000","last_undegraded":"2024-09-16T11:16:57.968679+0000","last_fullsized":"2024-09-16T11:16:57.968679+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:29.212441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"205'491","reported_seq":8704,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968268+0000","last_change":"2024-09-16T11:16:47.486651+0000","last_active":"2024-09-16T11:16:57.968268+0000","last_peered":"2024-09-16T11:16:57.968268+0000","last_clean":"2024-09-16T11:16:57.968268+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:16:57.968268+0000","last_undegraded":"2024-09-16T11:16:57.968268+0000","last_fullsized":"2024-09-16T11:16:57.968268+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:45.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647126+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:16:57.647126+0000","last_peered":"2024-09-16T11:16:57.647126+0000","last_clean":"2024-09-16T11:16:57.647126+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:16:57.647126+0000","last_undegraded":"2024-09-16T11:16:57.647126+0000","last_fullsized":"2024-09-16T11:16:57.647126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648781+0000","last_change":"2024-09-16T11:16:43.567954+0000","last_active":"2024-09-16T11:16:57.648781+0000","last_peered":"2024-09-16T11:16:57.648781+0000","last_clean":"2024-09-16T11:16:57.648781+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:16:57.648781+0000","last_undegraded":"2024-09-16T11:16:57.648781+0000","last_fullsized":"2024-09-16T11:16:57.648781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:22:11.421317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"206'644","reported_seq":6808,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968521+0000","last_change":"2024-09-16T11:16:47.483421+0000","last_active":"2024-09-16T11:16:57.968521+0000","last_peered":"2024-09-16T11:16:57.968521+0000","last_clean":"2024-09-16T11:16:57.968521+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:16:57.968521+0000","last_undegraded":"2024-09-16T11:16:57.968521+0000","last_fullsized":"2024-09-16T11:16:57.968521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:13.778395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":168,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.967896+0000","last_change":"2024-09-16T11:16:47.481364+0000","last_active":"2024-09-16T11:16:57.967896+0000","last_peered":"2024-09-16T11:16:57.967896+0000","last_clean":"2024-09-16T11:16:57.967896+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:16:57.967896+0000","last_undegraded":"2024-09-16T11:16:57.967896+0000","last_fullsized":"2024-09-16T11:16:57.967896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:22.268248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969784+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:16:57.969784+0000","last_peered":"2024-09-16T11:16:57.969784+0000","last_clean":"2024-09-16T11:16:57.969784+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:16:57.969784+0000","last_undegraded":"2024-09-16T11:16:57.969784+0000","last_fullsized":"2024-09-16T11:16:57.969784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"193'684","reported_seq":5002,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969111+0000","last_change":"2024-09-16T11:16:50.477765+0000","last_active":"2024-09-16T11:16:57.969111+0000","last_peered":"2024-09-16T11:16:57.969111+0000","last_clean":"2024-09-16T11:16:57.969111+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:16:57.969111+0000","last_undegraded":"2024-09-16T11:16:57.969111+0000","last_fullsized":"2024-09-16T11:16:57.969111+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:14.471034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"212'2","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644047+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:16:57.644047+0000","last_peered":"2024-09-16T11:16:57.644047+0000","last_clean":"2024-09-16T11:16:57.644047+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:16:57.644047+0000","last_undegraded":"2024-09-16T11:16:57.644047+0000","last_fullsized":"2024-09-16T11:16:57.644047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969075+0000","last_change":"2024-09-16T11:16:42.521626+0000","last_active":"2024-09-16T11:16:57.969075+0000","last_peered":"2024-09-16T11:16:57.969075+0000","last_clean":"2024-09-16T11:16:57.969075+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:16:57.969075+0000","last_undegraded":"2024-09-16T11:16:57.969075+0000","last_fullsized":"2024-09-16T11:16:57.969075+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:03.898472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"207'538","reported_seq":4282,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644466+0000","last_change":"2024-09-16T11:16:36.479810+0000","last_active":"2024-09-16T11:16:57.644466+0000","last_peered":"2024-09-16T11:16:57.644466+0000","last_clean":"2024-09-16T11:16:57.644466+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:16:57.644466+0000","last_undegraded":"2024-09-16T11:16:57.644466+0000","last_fullsized":"2024-09-16T11:16:57.644466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:09.533692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"168'4","reported_seq":289,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969684+0000","last_change":"2024-09-16T11:16:43.545938+0000","last_active":"2024-09-16T11:16:57.969684+0000","last_peered":"2024-09-16T11:16:57.969684+0000","last_clean":"2024-09-16T11:16:57.969684+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:16:57.969684+0000","last_undegraded":"2024-09-16T11:16:57.969684+0000","last_fullsized":"2024-09-16T11:16:57.969684+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:35.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648992+0000","last_change":"2024-09-16T11:16:43.565231+0000","last_active":"2024-09-16T11:16:57.648992+0000","last_peered":"2024-09-16T11:16:57.648992+0000","last_clean":"2024-09-16T11:16:57.648992+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:16:57.648992+0000","last_undegraded":"2024-09-16T11:16:57.648992+0000","last_fullsized":"2024-09-16T11:16:57.648992+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:45.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":30,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970000+0000","last_change":"2024-09-16T11:16:42.529324+0000","last_active":"2024-09-16T11:16:57.970000+0000","last_peered":"2024-09-16T11:16:57.970000+0000","last_clean":"2024-09-16T11:16:57.970000+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:16:57.970000+0000","last_undegraded":"2024-09-16T11:16:57.970000+0000","last_fullsized":"2024-09-16T11:16:57.970000+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:09.000275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"205'573","reported_seq":6327,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:33.633708+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"202'885","reported_seq":3505,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648499+0000","last_change":"2024-09-16T11:16:48.483747+0000","last_active":"2024-09-16T11:16:57.648499+0000","last_peered":"2024-09-16T11:16:57.648499+0000","last_clean":"2024-09-16T11:16:57.648499+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:16:57.648499+0000","last_undegraded":"2024-09-16T11:16:57.648499+0000","last_fullsized":"2024-09-16T11:16:57.648499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11038,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646703+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:57.646703+0000","last_peered":"2024-09-16T11:16:57.646703+0000","last_clean":"2024-09-16T11:16:57.646703+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:57.646703+0000","last_undegraded":"2024-09-16T11:16:57.646703+0000","last_fullsized":"2024-09-16T11:16:57.646703+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:08.661865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"205'573","reported_seq":5145,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:37.481009+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:45.260823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"212'668","reported_seq":8071,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644042+0000","last_change":"2024-09-16T11:16:39.501093+0000","last_active":"2024-09-16T11:16:57.644042+0000","last_peered":"2024-09-16T11:16:57.644042+0000","last_clean":"2024-09-16T11:16:57.644042+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:16:57.644042+0000","last_undegraded":"2024-09-16T11:16:57.644042+0000","last_fullsized":"2024-09-16T11:16:57.644042+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:46.228140+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"206'1205","reported_seq":10338,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646826+0000","last_change":"2024-09-16T11:16:33.632197+0000","last_active":"2024-09-16T11:16:57.646826+0000","last_peered":"2024-09-16T11:16:57.646826+0000","last_clean":"2024-09-16T11:16:57.646826+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:16:57.646826+0000","last_undegraded":"2024-09-16T11:16:57.646826+0000","last_fullsized":"2024-09-16T11:16:57.646826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17294,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647063+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:57.647063+0000","last_peered":"2024-09-16T11:16:57.647063+0000","last_clean":"2024-09-16T11:16:57.647063+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:57.647063+0000","last_undegraded":"2024-09-16T11:16:57.647063+0000","last_fullsized":"2024-09-16T11:16:57.647063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"200'769","reported_seq":8815,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647188+0000","last_change":"2024-09-16T11:16:33.635099+0000","last_active":"2024-09-16T11:16:57.647188+0000","last_peered":"2024-09-16T11:16:57.647188+0000","last_clean":"2024-09-16T11:16:57.647188+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:16:57.647188+0000","last_undegraded":"2024-09-16T11:16:57.647188+0000","last_fullsized":"2024-09-16T11:16:57.647188+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"200'540","reported_seq":5263,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969308+0000","last_change":"2024-09-16T11:16:47.488023+0000","last_active":"2024-09-16T11:16:57.969308+0000","last_peered":"2024-09-16T11:16:57.969308+0000","last_clean":"2024-09-16T11:16:57.969308+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:16:57.969308+0000","last_undegraded":"2024-09-16T11:16:57.969308+0000","last_fullsized":"2024-09-16T11:16:57.969308+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:54.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23473,"ondisk_log_size":23473,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034055,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305468,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066372,"statfs":{"total":96636764160,"available":96323964928,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300092,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071748,"statfs":{"total":96636764160,"available":96329469952,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.0600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":1.296},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.999}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":1.1060000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.2390000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.403},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.462},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.879}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.085},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.157}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.002}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.052},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.286}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312580,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059260,"statfs":{"total":96636764160,"available":96316682240,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.139,"5min":1.329,"15min":1.329},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1379999999999999,"5min":1.173,"15min":1.173},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86699999999999999,"15min":0.90100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.874,"15min":0.91800000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.026,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0029999999999999,"15min":1.0469999999999999},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.304,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.278}]},{"osd":3,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0649999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.58299999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1120000000000001,"5min":2.077,"15min":2.077},"last":1.3129999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2829999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.417},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0980000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.069,"15min":1.1299999999999999},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.256,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.119,"15min":1.1419999999999999},"min":{"1min":0.627,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.355,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.508}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.456},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.6399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.503},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2785280,"data_stored":1427036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":2.0720000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":2.1520000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.498},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308412,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063428,"statfs":{"total":96636764160,"available":96320950272,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":1.0980000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.212},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":1.371}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":1.0029999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.4530000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":1.4039999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313148,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058692,"statfs":{"total":96636764160,"available":96316100608,"internally_reserved":0,"allocated":3383296,"data_stored":2017487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.036}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.27},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":1.3160000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.22},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.8400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:02.134 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T11:17:02.134 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T11:17:02.425 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T11:17:02.434 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-16T11:17:02.434 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:17:02.714 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:02.714 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":231,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:16:58.650867+0000","last_up_change":"2024-09-16T11:16:40.476259+0000","last_in_change":"2024-09-16T11:16:42.499018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:12:53.361644+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:13:02.524586+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:13:11.876887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":87,"snap_epoch":206,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":181,"up_from":213,"up_thru":218,"down_at":182,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":32265},{"type":"v1","addr":"172.21.15.165:6820","nonce":32265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":32265},{"type":"v1","addr":"172.21.15.165:6825","nonce":32265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":32265},{"type":"v1","addr":"172.21.15.165:6831","nonce":32265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":32265},{"type":"v1","addr":"172.21.15.165:6829","nonce":32265}]},"public_addr":"172.21.15.165:6820/32265","cluster_addr":"172.21.15.165:6825/32265","heartbeat_back_addr":"172.21.15.165:6831/32265","heartbeat_front_addr":"172.21.15.165:6829/32265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":182}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:17:02.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:17:02.728 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:03.022 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:03.023 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:03.033 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T11:17:02.014261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23554,"ondisk_log_size":23554,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444152,"kb_used_data":23028,"kb_used_omap":48,"kb_used_meta":2420943,"kb_avail":752530568,"statfs":{"total":773094113280,"available":770591301632,"internally_reserved":0,"allocated":23580672,"data_stored":12634200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2479045869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001566"},"pg_stats":[{"pgid":"4.1f","version":"206'498","reported_seq":7631,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969898+0000","last_change":"2024-09-16T11:16:46.480011+0000","last_active":"2024-09-16T11:16:57.969898+0000","last_peered":"2024-09-16T11:16:57.969898+0000","last_clean":"2024-09-16T11:16:57.969898+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:16:57.969898+0000","last_undegraded":"2024-09-16T11:16:57.969898+0000","last_fullsized":"2024-09-16T11:16:57.969898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:17.579128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12720,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644207+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:57.644207+0000","last_peered":"2024-09-16T11:16:57.644207+0000","last_clean":"2024-09-16T11:16:57.644207+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:57.644207+0000","last_undegraded":"2024-09-16T11:16:57.644207+0000","last_fullsized":"2024-09-16T11:16:57.644207+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:05.159446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"205'510","reported_seq":5433,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970493+0000","last_change":"2024-09-16T11:16:34.642934+0000","last_active":"2024-09-16T11:16:57.970493+0000","last_peered":"2024-09-16T11:16:57.970493+0000","last_clean":"2024-09-16T11:16:57.970493+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:16:57.970493+0000","last_undegraded":"2024-09-16T11:16:57.970493+0000","last_fullsized":"2024-09-16T11:16:57.970493+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:00.434118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"206'1074","reported_seq":9478,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969101+0000","last_change":"2024-09-16T11:16:38.490863+0000","last_active":"2024-09-16T11:16:57.969101+0000","last_peered":"2024-09-16T11:16:57.969101+0000","last_clean":"2024-09-16T11:16:57.969101+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:16:57.969101+0000","last_undegraded":"2024-09-16T11:16:57.969101+0000","last_fullsized":"2024-09-16T11:16:57.969101+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:07.899214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7521,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646819+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:57.646819+0000","last_peered":"2024-09-16T11:16:57.646819+0000","last_clean":"2024-09-16T11:16:57.646819+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:57.646819+0000","last_undegraded":"2024-09-16T11:16:57.646819+0000","last_fullsized":"2024-09-16T11:16:57.646819+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"206'701","reported_seq":8134,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646713+0000","last_change":"2024-09-16T11:16:33.634941+0000","last_active":"2024-09-16T11:16:57.646713+0000","last_peered":"2024-09-16T11:16:57.646713+0000","last_clean":"2024-09-16T11:16:57.646713+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:16:57.646713+0000","last_undegraded":"2024-09-16T11:16:57.646713+0000","last_fullsized":"2024-09-16T11:16:57.646713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"199'840","reported_seq":3434,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970116+0000","last_change":"2024-09-16T11:16:33.627164+0000","last_active":"2024-09-16T11:16:57.970116+0000","last_peered":"2024-09-16T11:16:57.970116+0000","last_clean":"2024-09-16T11:16:57.970116+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:16:57.970116+0000","last_undegraded":"2024-09-16T11:16:57.970116+0000","last_fullsized":"2024-09-16T11:16:57.970116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"206'775","reported_seq":5543,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646596+0000","last_change":"2024-09-16T11:16:33.633666+0000","last_active":"2024-09-16T11:16:57.646596+0000","last_peered":"2024-09-16T11:16:57.646596+0000","last_clean":"2024-09-16T11:16:57.646596+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:16:57.646596+0000","last_undegraded":"2024-09-16T11:16:57.646596+0000","last_fullsized":"2024-09-16T11:16:57.646596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7723,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969928+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:57.969928+0000","last_peered":"2024-09-16T11:16:57.969928+0000","last_clean":"2024-09-16T11:16:57.969928+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:57.969928+0000","last_undegraded":"2024-09-16T11:16:57.969928+0000","last_fullsized":"2024-09-16T11:16:57.969928+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:42.706001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"200'499","reported_seq":7620,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644530+0000","last_change":"2024-09-16T11:16:33.624591+0000","last_active":"2024-09-16T11:16:57.644530+0000","last_peered":"2024-09-16T11:16:57.644530+0000","last_clean":"2024-09-16T11:16:57.644530+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:16:57.644530+0000","last_undegraded":"2024-09-16T11:16:57.644530+0000","last_fullsized":"2024-09-16T11:16:57.644530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"205'486","reported_seq":4883,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648892+0000","last_change":"2024-09-16T11:16:45.480011+0000","last_active":"2024-09-16T11:16:57.648892+0000","last_peered":"2024-09-16T11:16:57.648892+0000","last_clean":"2024-09-16T11:16:57.648892+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:16:57.648892+0000","last_undegraded":"2024-09-16T11:16:57.648892+0000","last_fullsized":"2024-09-16T11:16:57.648892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:23.227245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"205'673","reported_seq":7416,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969848+0000","last_change":"2024-09-16T11:16:39.492405+0000","last_active":"2024-09-16T11:16:57.969848+0000","last_peered":"2024-09-16T11:16:57.969848+0000","last_clean":"2024-09-16T11:16:57.969848+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:16:57.969848+0000","last_undegraded":"2024-09-16T11:16:57.969848+0000","last_fullsized":"2024-09-16T11:16:57.969848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:21.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"205'593","reported_seq":4530,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647123+0000","last_change":"2024-09-16T11:16:33.633503+0000","last_active":"2024-09-16T11:16:57.647123+0000","last_peered":"2024-09-16T11:16:57.647123+0000","last_clean":"2024-09-16T11:16:57.647123+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:16:57.647123+0000","last_undegraded":"2024-09-16T11:16:57.647123+0000","last_fullsized":"2024-09-16T11:16:57.647123+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"206'531","reported_seq":6108,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968113+0000","last_change":"2024-09-16T11:16:47.482430+0000","last_active":"2024-09-16T11:16:57.968113+0000","last_peered":"2024-09-16T11:16:57.968113+0000","last_clean":"2024-09-16T11:16:57.968113+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:16:57.968113+0000","last_undegraded":"2024-09-16T11:16:57.968113+0000","last_fullsized":"2024-09-16T11:16:57.968113+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:41.497485+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8207,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969889+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:57.969889+0000","last_peered":"2024-09-16T11:16:57.969889+0000","last_clean":"2024-09-16T11:16:57.969889+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:57.969889+0000","last_undegraded":"2024-09-16T11:16:57.969889+0000","last_fullsized":"2024-09-16T11:16:57.969889+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:05.400722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"193'627","reported_seq":7679,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969692+0000","last_change":"2024-09-16T11:16:35.483110+0000","last_active":"2024-09-16T11:16:57.969692+0000","last_peered":"2024-09-16T11:16:57.969692+0000","last_clean":"2024-09-16T11:16:57.969692+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:16:57.969692+0000","last_undegraded":"2024-09-16T11:16:57.969692+0000","last_fullsized":"2024-09-16T11:16:57.969692+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:27.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969871+0000","last_change":"2024-09-16T11:16:42.522836+0000","last_active":"2024-09-16T11:16:57.969871+0000","last_peered":"2024-09-16T11:16:57.969871+0000","last_clean":"2024-09-16T11:16:57.969871+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:16:57.969871+0000","last_undegraded":"2024-09-16T11:16:57.969871+0000","last_fullsized":"2024-09-16T11:16:57.969871+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:31.057363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"200'1007","reported_seq":6373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646875+0000","last_change":"2024-09-16T11:16:46.477020+0000","last_active":"2024-09-16T11:16:57.646875+0000","last_peered":"2024-09-16T11:16:57.646875+0000","last_clean":"2024-09-16T11:16:57.646875+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:16:57.646875+0000","last_undegraded":"2024-09-16T11:16:57.646875+0000","last_fullsized":"2024-09-16T11:16:57.646875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:46.468919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968883+0000","last_change":"2024-09-16T11:16:47.479923+0000","last_active":"2024-09-16T11:16:57.968883+0000","last_peered":"2024-09-16T11:16:57.968883+0000","last_clean":"2024-09-16T11:16:57.968883+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:16:57.968883+0000","last_undegraded":"2024-09-16T11:16:57.968883+0000","last_fullsized":"2024-09-16T11:16:57.968883+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:03.586218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6183,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970015+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:57.970015+0000","last_peered":"2024-09-16T11:16:57.970015+0000","last_clean":"2024-09-16T11:16:57.970015+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:57.970015+0000","last_undegraded":"2024-09-16T11:16:57.970015+0000","last_fullsized":"2024-09-16T11:16:57.970015+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:31.708609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969274+0000","last_change":"2024-09-16T11:16:42.520140+0000","last_active":"2024-09-16T11:16:57.969274+0000","last_peered":"2024-09-16T11:16:57.969274+0000","last_clean":"2024-09-16T11:16:57.969274+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:16:57.969274+0000","last_undegraded":"2024-09-16T11:16:57.969274+0000","last_fullsized":"2024-09-16T11:16:57.969274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:01.926666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"207'1204","reported_seq":18994,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644171+0000","last_change":"2024-09-16T11:16:47.485172+0000","last_active":"2024-09-16T11:16:57.644171+0000","last_peered":"2024-09-16T11:16:57.644171+0000","last_clean":"2024-09-16T11:16:57.644171+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:16:57.644171+0000","last_undegraded":"2024-09-16T11:16:57.644171+0000","last_fullsized":"2024-09-16T11:16:57.644171+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:10.056185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968679+0000","last_change":"2024-09-16T11:16:47.480778+0000","last_active":"2024-09-16T11:16:57.968679+0000","last_peered":"2024-09-16T11:16:57.968679+0000","last_clean":"2024-09-16T11:16:57.968679+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:16:57.968679+0000","last_undegraded":"2024-09-16T11:16:57.968679+0000","last_fullsized":"2024-09-16T11:16:57.968679+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:29.212441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"205'491","reported_seq":8704,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968268+0000","last_change":"2024-09-16T11:16:47.486651+0000","last_active":"2024-09-16T11:16:57.968268+0000","last_peered":"2024-09-16T11:16:57.968268+0000","last_clean":"2024-09-16T11:16:57.968268+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:16:57.968268+0000","last_undegraded":"2024-09-16T11:16:57.968268+0000","last_fullsized":"2024-09-16T11:16:57.968268+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:45.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647126+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:16:57.647126+0000","last_peered":"2024-09-16T11:16:57.647126+0000","last_clean":"2024-09-16T11:16:57.647126+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:16:57.647126+0000","last_undegraded":"2024-09-16T11:16:57.647126+0000","last_fullsized":"2024-09-16T11:16:57.647126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648781+0000","last_change":"2024-09-16T11:16:43.567954+0000","last_active":"2024-09-16T11:16:57.648781+0000","last_peered":"2024-09-16T11:16:57.648781+0000","last_clean":"2024-09-16T11:16:57.648781+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:16:57.648781+0000","last_undegraded":"2024-09-16T11:16:57.648781+0000","last_fullsized":"2024-09-16T11:16:57.648781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:22:11.421317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"206'644","reported_seq":6808,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968521+0000","last_change":"2024-09-16T11:16:47.483421+0000","last_active":"2024-09-16T11:16:57.968521+0000","last_peered":"2024-09-16T11:16:57.968521+0000","last_clean":"2024-09-16T11:16:57.968521+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:16:57.968521+0000","last_undegraded":"2024-09-16T11:16:57.968521+0000","last_fullsized":"2024-09-16T11:16:57.968521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:13.778395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":168,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.967896+0000","last_change":"2024-09-16T11:16:47.481364+0000","last_active":"2024-09-16T11:16:57.967896+0000","last_peered":"2024-09-16T11:16:57.967896+0000","last_clean":"2024-09-16T11:16:57.967896+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:16:57.967896+0000","last_undegraded":"2024-09-16T11:16:57.967896+0000","last_fullsized":"2024-09-16T11:16:57.967896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:22.268248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969784+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:16:57.969784+0000","last_peered":"2024-09-16T11:16:57.969784+0000","last_clean":"2024-09-16T11:16:57.969784+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:16:57.969784+0000","last_undegraded":"2024-09-16T11:16:57.969784+0000","last_fullsized":"2024-09-16T11:16:57.969784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"193'684","reported_seq":5002,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969111+0000","last_change":"2024-09-16T11:16:50.477765+0000","last_active":"2024-09-16T11:16:57.969111+0000","last_peered":"2024-09-16T11:16:57.969111+0000","last_clean":"2024-09-16T11:16:57.969111+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:16:57.969111+0000","last_undegraded":"2024-09-16T11:16:57.969111+0000","last_fullsized":"2024-09-16T11:16:57.969111+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:14.471034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"212'2","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644047+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:16:57.644047+0000","last_peered":"2024-09-16T11:16:57.644047+0000","last_clean":"2024-09-16T11:16:57.644047+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:16:57.644047+0000","last_undegraded":"2024-09-16T11:16:57.644047+0000","last_fullsized":"2024-09-16T11:16:57.644047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969075+0000","last_change":"2024-09-16T11:16:42.521626+0000","last_active":"2024-09-16T11:16:57.969075+0000","last_peered":"2024-09-16T11:16:57.969075+0000","last_clean":"2024-09-16T11:16:57.969075+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:16:57.969075+0000","last_undegraded":"2024-09-16T11:16:57.969075+0000","last_fullsized":"2024-09-16T11:16:57.969075+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:03.898472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"207'538","reported_seq":4282,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644466+0000","last_change":"2024-09-16T11:16:36.479810+0000","last_active":"2024-09-16T11:16:57.644466+0000","last_peered":"2024-09-16T11:16:57.644466+0000","last_clean":"2024-09-16T11:16:57.644466+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:16:57.644466+0000","last_undegraded":"2024-09-16T11:16:57.644466+0000","last_fullsized":"2024-09-16T11:16:57.644466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:09.533692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"168'4","reported_seq":289,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969684+0000","last_change":"2024-09-16T11:16:43.545938+0000","last_active":"2024-09-16T11:16:57.969684+0000","last_peered":"2024-09-16T11:16:57.969684+0000","last_clean":"2024-09-16T11:16:57.969684+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:16:57.969684+0000","last_undegraded":"2024-09-16T11:16:57.969684+0000","last_fullsized":"2024-09-16T11:16:57.969684+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:35.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648992+0000","last_change":"2024-09-16T11:16:43.565231+0000","last_active":"2024-09-16T11:16:57.648992+0000","last_peered":"2024-09-16T11:16:57.648992+0000","last_clean":"2024-09-16T11:16:57.648992+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:16:57.648992+0000","last_undegraded":"2024-09-16T11:16:57.648992+0000","last_fullsized":"2024-09-16T11:16:57.648992+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:45.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":30,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970000+0000","last_change":"2024-09-16T11:16:42.529324+0000","last_active":"2024-09-16T11:16:57.970000+0000","last_peered":"2024-09-16T11:16:57.970000+0000","last_clean":"2024-09-16T11:16:57.970000+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:16:57.970000+0000","last_undegraded":"2024-09-16T11:16:57.970000+0000","last_fullsized":"2024-09-16T11:16:57.970000+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:09.000275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"205'573","reported_seq":6327,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:33.633708+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"202'885","reported_seq":3505,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648499+0000","last_change":"2024-09-16T11:16:48.483747+0000","last_active":"2024-09-16T11:16:57.648499+0000","last_peered":"2024-09-16T11:16:57.648499+0000","last_clean":"2024-09-16T11:16:57.648499+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:16:57.648499+0000","last_undegraded":"2024-09-16T11:16:57.648499+0000","last_fullsized":"2024-09-16T11:16:57.648499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11038,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646703+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:57.646703+0000","last_peered":"2024-09-16T11:16:57.646703+0000","last_clean":"2024-09-16T11:16:57.646703+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:57.646703+0000","last_undegraded":"2024-09-16T11:16:57.646703+0000","last_fullsized":"2024-09-16T11:16:57.646703+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:08.661865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"205'573","reported_seq":5145,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:37.481009+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:45.260823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"212'668","reported_seq":8071,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644042+0000","last_change":"2024-09-16T11:16:39.501093+0000","last_active":"2024-09-16T11:16:57.644042+0000","last_peered":"2024-09-16T11:16:57.644042+0000","last_clean":"2024-09-16T11:16:57.644042+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:16:57.644042+0000","last_undegraded":"2024-09-16T11:16:57.644042+0000","last_fullsized":"2024-09-16T11:16:57.644042+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:46.228140+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"206'1205","reported_seq":10338,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646826+0000","last_change":"2024-09-16T11:16:33.632197+0000","last_active":"2024-09-16T11:16:57.646826+0000","last_peered":"2024-09-16T11:16:57.646826+0000","last_clean":"2024-09-16T11:16:57.646826+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:16:57.646826+0000","last_undegraded":"2024-09-16T11:16:57.646826+0000","last_fullsized":"2024-09-16T11:16:57.646826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17294,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647063+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:57.647063+0000","last_peered":"2024-09-16T11:16:57.647063+0000","last_clean":"2024-09-16T11:16:57.647063+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:57.647063+0000","last_undegraded":"2024-09-16T11:16:57.647063+0000","last_fullsized":"2024-09-16T11:16:57.647063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"200'769","reported_seq":8815,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647188+0000","last_change":"2024-09-16T11:16:33.635099+0000","last_active":"2024-09-16T11:16:57.647188+0000","last_peered":"2024-09-16T11:16:57.647188+0000","last_clean":"2024-09-16T11:16:57.647188+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:16:57.647188+0000","last_undegraded":"2024-09-16T11:16:57.647188+0000","last_fullsized":"2024-09-16T11:16:57.647188+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"200'540","reported_seq":5263,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969308+0000","last_change":"2024-09-16T11:16:47.488023+0000","last_active":"2024-09-16T11:16:57.969308+0000","last_peered":"2024-09-16T11:16:57.969308+0000","last_clean":"2024-09-16T11:16:57.969308+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:16:57.969308+0000","last_undegraded":"2024-09-16T11:16:57.969308+0000","last_fullsized":"2024-09-16T11:16:57.969308+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:54.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23473,"ondisk_log_size":23473,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034055,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305468,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066372,"statfs":{"total":96636764160,"available":96323964928,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300092,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071748,"statfs":{"total":96636764160,"available":96329469952,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.0600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":1.296},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.999}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":1.1060000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.2390000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.403},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.462},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.879}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.085},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.157}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.002}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.052},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.286}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312580,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059260,"statfs":{"total":96636764160,"available":96316682240,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.139,"5min":1.329,"15min":1.329},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1379999999999999,"5min":1.173,"15min":1.173},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86699999999999999,"15min":0.90100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.874,"15min":0.91800000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.026,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0029999999999999,"15min":1.0469999999999999},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.304,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.278}]},{"osd":3,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0649999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.58299999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1120000000000001,"5min":2.077,"15min":2.077},"last":1.3129999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2829999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.417},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0980000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.069,"15min":1.1299999999999999},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.256,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.119,"15min":1.1419999999999999},"min":{"1min":0.627,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.355,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.508}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.456},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.6399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.503},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2785280,"data_stored":1427036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":2.0720000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":2.1520000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.498},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308412,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063428,"statfs":{"total":96636764160,"available":96320950272,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":1.0980000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.212},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":1.371}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":1.0029999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.4530000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":1.4039999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313148,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058692,"statfs":{"total":96636764160,"available":96316100608,"internally_reserved":0,"allocated":3383296,"data_stored":2017487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.036}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.27},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":1.3160000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.22},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.8400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:03.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:03.336 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:03.340 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:03.356 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T11:17:02.014261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23554,"ondisk_log_size":23554,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444152,"kb_used_data":23028,"kb_used_omap":48,"kb_used_meta":2420943,"kb_avail":752530568,"statfs":{"total":773094113280,"available":770591301632,"internally_reserved":0,"allocated":23580672,"data_stored":12634200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2479045869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001566"},"pg_stats":[{"pgid":"4.1f","version":"206'498","reported_seq":7631,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969898+0000","last_change":"2024-09-16T11:16:46.480011+0000","last_active":"2024-09-16T11:16:57.969898+0000","last_peered":"2024-09-16T11:16:57.969898+0000","last_clean":"2024-09-16T11:16:57.969898+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:16:57.969898+0000","last_undegraded":"2024-09-16T11:16:57.969898+0000","last_fullsized":"2024-09-16T11:16:57.969898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:17.579128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12720,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644207+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:57.644207+0000","last_peered":"2024-09-16T11:16:57.644207+0000","last_clean":"2024-09-16T11:16:57.644207+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:57.644207+0000","last_undegraded":"2024-09-16T11:16:57.644207+0000","last_fullsized":"2024-09-16T11:16:57.644207+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:05.159446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"205'510","reported_seq":5433,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970493+0000","last_change":"2024-09-16T11:16:34.642934+0000","last_active":"2024-09-16T11:16:57.970493+0000","last_peered":"2024-09-16T11:16:57.970493+0000","last_clean":"2024-09-16T11:16:57.970493+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:16:57.970493+0000","last_undegraded":"2024-09-16T11:16:57.970493+0000","last_fullsized":"2024-09-16T11:16:57.970493+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:00.434118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"206'1074","reported_seq":9478,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969101+0000","last_change":"2024-09-16T11:16:38.490863+0000","last_active":"2024-09-16T11:16:57.969101+0000","last_peered":"2024-09-16T11:16:57.969101+0000","last_clean":"2024-09-16T11:16:57.969101+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:16:57.969101+0000","last_undegraded":"2024-09-16T11:16:57.969101+0000","last_fullsized":"2024-09-16T11:16:57.969101+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:07.899214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7521,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646819+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:57.646819+0000","last_peered":"2024-09-16T11:16:57.646819+0000","last_clean":"2024-09-16T11:16:57.646819+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:57.646819+0000","last_undegraded":"2024-09-16T11:16:57.646819+0000","last_fullsized":"2024-09-16T11:16:57.646819+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"206'701","reported_seq":8134,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646713+0000","last_change":"2024-09-16T11:16:33.634941+0000","last_active":"2024-09-16T11:16:57.646713+0000","last_peered":"2024-09-16T11:16:57.646713+0000","last_clean":"2024-09-16T11:16:57.646713+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:16:57.646713+0000","last_undegraded":"2024-09-16T11:16:57.646713+0000","last_fullsized":"2024-09-16T11:16:57.646713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"199'840","reported_seq":3434,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970116+0000","last_change":"2024-09-16T11:16:33.627164+0000","last_active":"2024-09-16T11:16:57.970116+0000","last_peered":"2024-09-16T11:16:57.970116+0000","last_clean":"2024-09-16T11:16:57.970116+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:16:57.970116+0000","last_undegraded":"2024-09-16T11:16:57.970116+0000","last_fullsized":"2024-09-16T11:16:57.970116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"206'775","reported_seq":5543,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646596+0000","last_change":"2024-09-16T11:16:33.633666+0000","last_active":"2024-09-16T11:16:57.646596+0000","last_peered":"2024-09-16T11:16:57.646596+0000","last_clean":"2024-09-16T11:16:57.646596+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:16:57.646596+0000","last_undegraded":"2024-09-16T11:16:57.646596+0000","last_fullsized":"2024-09-16T11:16:57.646596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7723,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969928+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:57.969928+0000","last_peered":"2024-09-16T11:16:57.969928+0000","last_clean":"2024-09-16T11:16:57.969928+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:57.969928+0000","last_undegraded":"2024-09-16T11:16:57.969928+0000","last_fullsized":"2024-09-16T11:16:57.969928+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:42.706001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"200'499","reported_seq":7620,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644530+0000","last_change":"2024-09-16T11:16:33.624591+0000","last_active":"2024-09-16T11:16:57.644530+0000","last_peered":"2024-09-16T11:16:57.644530+0000","last_clean":"2024-09-16T11:16:57.644530+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:16:57.644530+0000","last_undegraded":"2024-09-16T11:16:57.644530+0000","last_fullsized":"2024-09-16T11:16:57.644530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"205'486","reported_seq":4883,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648892+0000","last_change":"2024-09-16T11:16:45.480011+0000","last_active":"2024-09-16T11:16:57.648892+0000","last_peered":"2024-09-16T11:16:57.648892+0000","last_clean":"2024-09-16T11:16:57.648892+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:16:57.648892+0000","last_undegraded":"2024-09-16T11:16:57.648892+0000","last_fullsized":"2024-09-16T11:16:57.648892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:23.227245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"205'673","reported_seq":7416,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969848+0000","last_change":"2024-09-16T11:16:39.492405+0000","last_active":"2024-09-16T11:16:57.969848+0000","last_peered":"2024-09-16T11:16:57.969848+0000","last_clean":"2024-09-16T11:16:57.969848+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:16:57.969848+0000","last_undegraded":"2024-09-16T11:16:57.969848+0000","last_fullsized":"2024-09-16T11:16:57.969848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:21.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"205'593","reported_seq":4530,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647123+0000","last_change":"2024-09-16T11:16:33.633503+0000","last_active":"2024-09-16T11:16:57.647123+0000","last_peered":"2024-09-16T11:16:57.647123+0000","last_clean":"2024-09-16T11:16:57.647123+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:16:57.647123+0000","last_undegraded":"2024-09-16T11:16:57.647123+0000","last_fullsized":"2024-09-16T11:16:57.647123+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"206'531","reported_seq":6108,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968113+0000","last_change":"2024-09-16T11:16:47.482430+0000","last_active":"2024-09-16T11:16:57.968113+0000","last_peered":"2024-09-16T11:16:57.968113+0000","last_clean":"2024-09-16T11:16:57.968113+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:16:57.968113+0000","last_undegraded":"2024-09-16T11:16:57.968113+0000","last_fullsized":"2024-09-16T11:16:57.968113+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:41.497485+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8207,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969889+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:57.969889+0000","last_peered":"2024-09-16T11:16:57.969889+0000","last_clean":"2024-09-16T11:16:57.969889+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:57.969889+0000","last_undegraded":"2024-09-16T11:16:57.969889+0000","last_fullsized":"2024-09-16T11:16:57.969889+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:05.400722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"193'627","reported_seq":7679,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969692+0000","last_change":"2024-09-16T11:16:35.483110+0000","last_active":"2024-09-16T11:16:57.969692+0000","last_peered":"2024-09-16T11:16:57.969692+0000","last_clean":"2024-09-16T11:16:57.969692+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:16:57.969692+0000","last_undegraded":"2024-09-16T11:16:57.969692+0000","last_fullsized":"2024-09-16T11:16:57.969692+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:27.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969871+0000","last_change":"2024-09-16T11:16:42.522836+0000","last_active":"2024-09-16T11:16:57.969871+0000","last_peered":"2024-09-16T11:16:57.969871+0000","last_clean":"2024-09-16T11:16:57.969871+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:16:57.969871+0000","last_undegraded":"2024-09-16T11:16:57.969871+0000","last_fullsized":"2024-09-16T11:16:57.969871+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:31.057363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"200'1007","reported_seq":6373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646875+0000","last_change":"2024-09-16T11:16:46.477020+0000","last_active":"2024-09-16T11:16:57.646875+0000","last_peered":"2024-09-16T11:16:57.646875+0000","last_clean":"2024-09-16T11:16:57.646875+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:16:57.646875+0000","last_undegraded":"2024-09-16T11:16:57.646875+0000","last_fullsized":"2024-09-16T11:16:57.646875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:46.468919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968883+0000","last_change":"2024-09-16T11:16:47.479923+0000","last_active":"2024-09-16T11:16:57.968883+0000","last_peered":"2024-09-16T11:16:57.968883+0000","last_clean":"2024-09-16T11:16:57.968883+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:16:57.968883+0000","last_undegraded":"2024-09-16T11:16:57.968883+0000","last_fullsized":"2024-09-16T11:16:57.968883+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:03.586218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6183,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970015+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:57.970015+0000","last_peered":"2024-09-16T11:16:57.970015+0000","last_clean":"2024-09-16T11:16:57.970015+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:57.970015+0000","last_undegraded":"2024-09-16T11:16:57.970015+0000","last_fullsized":"2024-09-16T11:16:57.970015+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:31.708609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969274+0000","last_change":"2024-09-16T11:16:42.520140+0000","last_active":"2024-09-16T11:16:57.969274+0000","last_peered":"2024-09-16T11:16:57.969274+0000","last_clean":"2024-09-16T11:16:57.969274+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:16:57.969274+0000","last_undegraded":"2024-09-16T11:16:57.969274+0000","last_fullsized":"2024-09-16T11:16:57.969274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:01.926666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"207'1204","reported_seq":18994,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644171+0000","last_change":"2024-09-16T11:16:47.485172+0000","last_active":"2024-09-16T11:16:57.644171+0000","last_peered":"2024-09-16T11:16:57.644171+0000","last_clean":"2024-09-16T11:16:57.644171+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:16:57.644171+0000","last_undegraded":"2024-09-16T11:16:57.644171+0000","last_fullsized":"2024-09-16T11:16:57.644171+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:10.056185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968679+0000","last_change":"2024-09-16T11:16:47.480778+0000","last_active":"2024-09-16T11:16:57.968679+0000","last_peered":"2024-09-16T11:16:57.968679+0000","last_clean":"2024-09-16T11:16:57.968679+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:16:57.968679+0000","last_undegraded":"2024-09-16T11:16:57.968679+0000","last_fullsized":"2024-09-16T11:16:57.968679+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:29.212441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"205'491","reported_seq":8704,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968268+0000","last_change":"2024-09-16T11:16:47.486651+0000","last_active":"2024-09-16T11:16:57.968268+0000","last_peered":"2024-09-16T11:16:57.968268+0000","last_clean":"2024-09-16T11:16:57.968268+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:16:57.968268+0000","last_undegraded":"2024-09-16T11:16:57.968268+0000","last_fullsized":"2024-09-16T11:16:57.968268+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:45.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647126+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:16:57.647126+0000","last_peered":"2024-09-16T11:16:57.647126+0000","last_clean":"2024-09-16T11:16:57.647126+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:16:57.647126+0000","last_undegraded":"2024-09-16T11:16:57.647126+0000","last_fullsized":"2024-09-16T11:16:57.647126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648781+0000","last_change":"2024-09-16T11:16:43.567954+0000","last_active":"2024-09-16T11:16:57.648781+0000","last_peered":"2024-09-16T11:16:57.648781+0000","last_clean":"2024-09-16T11:16:57.648781+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:16:57.648781+0000","last_undegraded":"2024-09-16T11:16:57.648781+0000","last_fullsized":"2024-09-16T11:16:57.648781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:22:11.421317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"206'644","reported_seq":6808,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968521+0000","last_change":"2024-09-16T11:16:47.483421+0000","last_active":"2024-09-16T11:16:57.968521+0000","last_peered":"2024-09-16T11:16:57.968521+0000","last_clean":"2024-09-16T11:16:57.968521+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:16:57.968521+0000","last_undegraded":"2024-09-16T11:16:57.968521+0000","last_fullsized":"2024-09-16T11:16:57.968521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:13.778395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":168,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.967896+0000","last_change":"2024-09-16T11:16:47.481364+0000","last_active":"2024-09-16T11:16:57.967896+0000","last_peered":"2024-09-16T11:16:57.967896+0000","last_clean":"2024-09-16T11:16:57.967896+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:16:57.967896+0000","last_undegraded":"2024-09-16T11:16:57.967896+0000","last_fullsized":"2024-09-16T11:16:57.967896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:22.268248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969784+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:16:57.969784+0000","last_peered":"2024-09-16T11:16:57.969784+0000","last_clean":"2024-09-16T11:16:57.969784+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:16:57.969784+0000","last_undegraded":"2024-09-16T11:16:57.969784+0000","last_fullsized":"2024-09-16T11:16:57.969784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"193'684","reported_seq":5002,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969111+0000","last_change":"2024-09-16T11:16:50.477765+0000","last_active":"2024-09-16T11:16:57.969111+0000","last_peered":"2024-09-16T11:16:57.969111+0000","last_clean":"2024-09-16T11:16:57.969111+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:16:57.969111+0000","last_undegraded":"2024-09-16T11:16:57.969111+0000","last_fullsized":"2024-09-16T11:16:57.969111+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:14.471034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"212'2","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644047+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:16:57.644047+0000","last_peered":"2024-09-16T11:16:57.644047+0000","last_clean":"2024-09-16T11:16:57.644047+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:16:57.644047+0000","last_undegraded":"2024-09-16T11:16:57.644047+0000","last_fullsized":"2024-09-16T11:16:57.644047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969075+0000","last_change":"2024-09-16T11:16:42.521626+0000","last_active":"2024-09-16T11:16:57.969075+0000","last_peered":"2024-09-16T11:16:57.969075+0000","last_clean":"2024-09-16T11:16:57.969075+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:16:57.969075+0000","last_undegraded":"2024-09-16T11:16:57.969075+0000","last_fullsized":"2024-09-16T11:16:57.969075+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:03.898472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"207'538","reported_seq":4282,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644466+0000","last_change":"2024-09-16T11:16:36.479810+0000","last_active":"2024-09-16T11:16:57.644466+0000","last_peered":"2024-09-16T11:16:57.644466+0000","last_clean":"2024-09-16T11:16:57.644466+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:16:57.644466+0000","last_undegraded":"2024-09-16T11:16:57.644466+0000","last_fullsized":"2024-09-16T11:16:57.644466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:09.533692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"168'4","reported_seq":289,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969684+0000","last_change":"2024-09-16T11:16:43.545938+0000","last_active":"2024-09-16T11:16:57.969684+0000","last_peered":"2024-09-16T11:16:57.969684+0000","last_clean":"2024-09-16T11:16:57.969684+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:16:57.969684+0000","last_undegraded":"2024-09-16T11:16:57.969684+0000","last_fullsized":"2024-09-16T11:16:57.969684+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:35.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648992+0000","last_change":"2024-09-16T11:16:43.565231+0000","last_active":"2024-09-16T11:16:57.648992+0000","last_peered":"2024-09-16T11:16:57.648992+0000","last_clean":"2024-09-16T11:16:57.648992+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:16:57.648992+0000","last_undegraded":"2024-09-16T11:16:57.648992+0000","last_fullsized":"2024-09-16T11:16:57.648992+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:45.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":30,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970000+0000","last_change":"2024-09-16T11:16:42.529324+0000","last_active":"2024-09-16T11:16:57.970000+0000","last_peered":"2024-09-16T11:16:57.970000+0000","last_clean":"2024-09-16T11:16:57.970000+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:16:57.970000+0000","last_undegraded":"2024-09-16T11:16:57.970000+0000","last_fullsized":"2024-09-16T11:16:57.970000+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:09.000275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"205'573","reported_seq":6327,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:33.633708+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"202'885","reported_seq":3505,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648499+0000","last_change":"2024-09-16T11:16:48.483747+0000","last_active":"2024-09-16T11:16:57.648499+0000","last_peered":"2024-09-16T11:16:57.648499+0000","last_clean":"2024-09-16T11:16:57.648499+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:16:57.648499+0000","last_undegraded":"2024-09-16T11:16:57.648499+0000","last_fullsized":"2024-09-16T11:16:57.648499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11038,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646703+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:57.646703+0000","last_peered":"2024-09-16T11:16:57.646703+0000","last_clean":"2024-09-16T11:16:57.646703+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:57.646703+0000","last_undegraded":"2024-09-16T11:16:57.646703+0000","last_fullsized":"2024-09-16T11:16:57.646703+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:08.661865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"205'573","reported_seq":5145,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:37.481009+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:45.260823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"212'668","reported_seq":8071,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644042+0000","last_change":"2024-09-16T11:16:39.501093+0000","last_active":"2024-09-16T11:16:57.644042+0000","last_peered":"2024-09-16T11:16:57.644042+0000","last_clean":"2024-09-16T11:16:57.644042+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:16:57.644042+0000","last_undegraded":"2024-09-16T11:16:57.644042+0000","last_fullsized":"2024-09-16T11:16:57.644042+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:46.228140+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"206'1205","reported_seq":10338,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646826+0000","last_change":"2024-09-16T11:16:33.632197+0000","last_active":"2024-09-16T11:16:57.646826+0000","last_peered":"2024-09-16T11:16:57.646826+0000","last_clean":"2024-09-16T11:16:57.646826+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:16:57.646826+0000","last_undegraded":"2024-09-16T11:16:57.646826+0000","last_fullsized":"2024-09-16T11:16:57.646826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17294,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647063+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:57.647063+0000","last_peered":"2024-09-16T11:16:57.647063+0000","last_clean":"2024-09-16T11:16:57.647063+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:57.647063+0000","last_undegraded":"2024-09-16T11:16:57.647063+0000","last_fullsized":"2024-09-16T11:16:57.647063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"200'769","reported_seq":8815,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647188+0000","last_change":"2024-09-16T11:16:33.635099+0000","last_active":"2024-09-16T11:16:57.647188+0000","last_peered":"2024-09-16T11:16:57.647188+0000","last_clean":"2024-09-16T11:16:57.647188+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:16:57.647188+0000","last_undegraded":"2024-09-16T11:16:57.647188+0000","last_fullsized":"2024-09-16T11:16:57.647188+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"200'540","reported_seq":5263,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969308+0000","last_change":"2024-09-16T11:16:47.488023+0000","last_active":"2024-09-16T11:16:57.969308+0000","last_peered":"2024-09-16T11:16:57.969308+0000","last_clean":"2024-09-16T11:16:57.969308+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:16:57.969308+0000","last_undegraded":"2024-09-16T11:16:57.969308+0000","last_fullsized":"2024-09-16T11:16:57.969308+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:54.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23473,"ondisk_log_size":23473,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034055,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305468,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066372,"statfs":{"total":96636764160,"available":96323964928,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300092,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071748,"statfs":{"total":96636764160,"available":96329469952,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.0600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":1.296},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.999}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":1.1060000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.2390000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.403},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.462},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.879}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.085},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.157}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.002}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.052},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.286}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312580,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059260,"statfs":{"total":96636764160,"available":96316682240,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.139,"5min":1.329,"15min":1.329},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1379999999999999,"5min":1.173,"15min":1.173},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86699999999999999,"15min":0.90100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.874,"15min":0.91800000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.026,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0029999999999999,"15min":1.0469999999999999},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.304,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.278}]},{"osd":3,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0649999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.58299999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1120000000000001,"5min":2.077,"15min":2.077},"last":1.3129999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2829999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.417},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0980000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.069,"15min":1.1299999999999999},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.256,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.119,"15min":1.1419999999999999},"min":{"1min":0.627,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.355,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.508}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.456},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.6399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.503},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2785280,"data_stored":1427036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":2.0720000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":2.1520000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.498},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308412,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063428,"statfs":{"total":96636764160,"available":96320950272,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":1.0980000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.212},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":1.371}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":1.0029999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.4530000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":1.4039999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313148,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058692,"statfs":{"total":96636764160,"available":96316100608,"internally_reserved":0,"allocated":3383296,"data_stored":2017487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.036}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.27},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":1.3160000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.22},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.8400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:03.359 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:17:03.359 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:03.658 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:03.660 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:03.673 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T11:17:02.014261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23554,"ondisk_log_size":23554,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444152,"kb_used_data":23028,"kb_used_omap":48,"kb_used_meta":2420943,"kb_avail":752530568,"statfs":{"total":773094113280,"available":770591301632,"internally_reserved":0,"allocated":23580672,"data_stored":12634200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2479045869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001566"},"pg_stats":[{"pgid":"4.1f","version":"206'498","reported_seq":7631,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969898+0000","last_change":"2024-09-16T11:16:46.480011+0000","last_active":"2024-09-16T11:16:57.969898+0000","last_peered":"2024-09-16T11:16:57.969898+0000","last_clean":"2024-09-16T11:16:57.969898+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:16:57.969898+0000","last_undegraded":"2024-09-16T11:16:57.969898+0000","last_fullsized":"2024-09-16T11:16:57.969898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:17.579128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12720,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644207+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:57.644207+0000","last_peered":"2024-09-16T11:16:57.644207+0000","last_clean":"2024-09-16T11:16:57.644207+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:57.644207+0000","last_undegraded":"2024-09-16T11:16:57.644207+0000","last_fullsized":"2024-09-16T11:16:57.644207+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:17:05.159446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"205'510","reported_seq":5433,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970493+0000","last_change":"2024-09-16T11:16:34.642934+0000","last_active":"2024-09-16T11:16:57.970493+0000","last_peered":"2024-09-16T11:16:57.970493+0000","last_clean":"2024-09-16T11:16:57.970493+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:16:57.970493+0000","last_undegraded":"2024-09-16T11:16:57.970493+0000","last_fullsized":"2024-09-16T11:16:57.970493+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:05:00.434118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"206'1074","reported_seq":9478,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969101+0000","last_change":"2024-09-16T11:16:38.490863+0000","last_active":"2024-09-16T11:16:57.969101+0000","last_peered":"2024-09-16T11:16:57.969101+0000","last_clean":"2024-09-16T11:16:57.969101+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:16:57.969101+0000","last_undegraded":"2024-09-16T11:16:57.969101+0000","last_fullsized":"2024-09-16T11:16:57.969101+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:12:07.899214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7521,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646819+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:57.646819+0000","last_peered":"2024-09-16T11:16:57.646819+0000","last_clean":"2024-09-16T11:16:57.646819+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:57.646819+0000","last_undegraded":"2024-09-16T11:16:57.646819+0000","last_fullsized":"2024-09-16T11:16:57.646819+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:09.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"206'701","reported_seq":8134,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646713+0000","last_change":"2024-09-16T11:16:33.634941+0000","last_active":"2024-09-16T11:16:57.646713+0000","last_peered":"2024-09-16T11:16:57.646713+0000","last_clean":"2024-09-16T11:16:57.646713+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:16:57.646713+0000","last_undegraded":"2024-09-16T11:16:57.646713+0000","last_fullsized":"2024-09-16T11:16:57.646713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:29.085351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"199'840","reported_seq":3434,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970116+0000","last_change":"2024-09-16T11:16:33.627164+0000","last_active":"2024-09-16T11:16:57.970116+0000","last_peered":"2024-09-16T11:16:57.970116+0000","last_clean":"2024-09-16T11:16:57.970116+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:16:57.970116+0000","last_undegraded":"2024-09-16T11:16:57.970116+0000","last_fullsized":"2024-09-16T11:16:57.970116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:16.562502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"206'775","reported_seq":5543,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646596+0000","last_change":"2024-09-16T11:16:33.633666+0000","last_active":"2024-09-16T11:16:57.646596+0000","last_peered":"2024-09-16T11:16:57.646596+0000","last_clean":"2024-09-16T11:16:57.646596+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:16:57.646596+0000","last_undegraded":"2024-09-16T11:16:57.646596+0000","last_fullsized":"2024-09-16T11:16:57.646596+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:06:08.670840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7723,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969928+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:57.969928+0000","last_peered":"2024-09-16T11:16:57.969928+0000","last_clean":"2024-09-16T11:16:57.969928+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:57.969928+0000","last_undegraded":"2024-09-16T11:16:57.969928+0000","last_fullsized":"2024-09-16T11:16:57.969928+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:14:42.706001+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"200'499","reported_seq":7620,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644530+0000","last_change":"2024-09-16T11:16:33.624591+0000","last_active":"2024-09-16T11:16:57.644530+0000","last_peered":"2024-09-16T11:16:57.644530+0000","last_clean":"2024-09-16T11:16:57.644530+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:16:57.644530+0000","last_undegraded":"2024-09-16T11:16:57.644530+0000","last_fullsized":"2024-09-16T11:16:57.644530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:49.932537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"205'486","reported_seq":4883,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648892+0000","last_change":"2024-09-16T11:16:45.480011+0000","last_active":"2024-09-16T11:16:57.648892+0000","last_peered":"2024-09-16T11:16:57.648892+0000","last_clean":"2024-09-16T11:16:57.648892+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:16:57.648892+0000","last_undegraded":"2024-09-16T11:16:57.648892+0000","last_fullsized":"2024-09-16T11:16:57.648892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:23.227245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"205'673","reported_seq":7416,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969848+0000","last_change":"2024-09-16T11:16:39.492405+0000","last_active":"2024-09-16T11:16:57.969848+0000","last_peered":"2024-09-16T11:16:57.969848+0000","last_clean":"2024-09-16T11:16:57.969848+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:16:57.969848+0000","last_undegraded":"2024-09-16T11:16:57.969848+0000","last_fullsized":"2024-09-16T11:16:57.969848+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:21.207902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"205'593","reported_seq":4530,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647123+0000","last_change":"2024-09-16T11:16:33.633503+0000","last_active":"2024-09-16T11:16:57.647123+0000","last_peered":"2024-09-16T11:16:57.647123+0000","last_clean":"2024-09-16T11:16:57.647123+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:16:57.647123+0000","last_undegraded":"2024-09-16T11:16:57.647123+0000","last_fullsized":"2024-09-16T11:16:57.647123+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:07.042241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"206'531","reported_seq":6108,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968113+0000","last_change":"2024-09-16T11:16:47.482430+0000","last_active":"2024-09-16T11:16:57.968113+0000","last_peered":"2024-09-16T11:16:57.968113+0000","last_clean":"2024-09-16T11:16:57.968113+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:16:57.968113+0000","last_undegraded":"2024-09-16T11:16:57.968113+0000","last_fullsized":"2024-09-16T11:16:57.968113+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:41.497485+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8207,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969889+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:57.969889+0000","last_peered":"2024-09-16T11:16:57.969889+0000","last_clean":"2024-09-16T11:16:57.969889+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:57.969889+0000","last_undegraded":"2024-09-16T11:16:57.969889+0000","last_fullsized":"2024-09-16T11:16:57.969889+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:05.400722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"193'627","reported_seq":7679,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969692+0000","last_change":"2024-09-16T11:16:35.483110+0000","last_active":"2024-09-16T11:16:57.969692+0000","last_peered":"2024-09-16T11:16:57.969692+0000","last_clean":"2024-09-16T11:16:57.969692+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:16:57.969692+0000","last_undegraded":"2024-09-16T11:16:57.969692+0000","last_fullsized":"2024-09-16T11:16:57.969692+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:25:27.542973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969871+0000","last_change":"2024-09-16T11:16:42.522836+0000","last_active":"2024-09-16T11:16:57.969871+0000","last_peered":"2024-09-16T11:16:57.969871+0000","last_clean":"2024-09-16T11:16:57.969871+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:16:57.969871+0000","last_undegraded":"2024-09-16T11:16:57.969871+0000","last_fullsized":"2024-09-16T11:16:57.969871+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:31.057363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"200'1007","reported_seq":6373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646875+0000","last_change":"2024-09-16T11:16:46.477020+0000","last_active":"2024-09-16T11:16:57.646875+0000","last_peered":"2024-09-16T11:16:57.646875+0000","last_clean":"2024-09-16T11:16:57.646875+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:16:57.646875+0000","last_undegraded":"2024-09-16T11:16:57.646875+0000","last_fullsized":"2024-09-16T11:16:57.646875+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:46.468919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968883+0000","last_change":"2024-09-16T11:16:47.479923+0000","last_active":"2024-09-16T11:16:57.968883+0000","last_peered":"2024-09-16T11:16:57.968883+0000","last_clean":"2024-09-16T11:16:57.968883+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:16:57.968883+0000","last_undegraded":"2024-09-16T11:16:57.968883+0000","last_fullsized":"2024-09-16T11:16:57.968883+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:36:03.586218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6183,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970015+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:57.970015+0000","last_peered":"2024-09-16T11:16:57.970015+0000","last_clean":"2024-09-16T11:16:57.970015+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:57.970015+0000","last_undegraded":"2024-09-16T11:16:57.970015+0000","last_fullsized":"2024-09-16T11:16:57.970015+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:31.708609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969274+0000","last_change":"2024-09-16T11:16:42.520140+0000","last_active":"2024-09-16T11:16:57.969274+0000","last_peered":"2024-09-16T11:16:57.969274+0000","last_clean":"2024-09-16T11:16:57.969274+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:16:57.969274+0000","last_undegraded":"2024-09-16T11:16:57.969274+0000","last_fullsized":"2024-09-16T11:16:57.969274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:01.926666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"207'1204","reported_seq":18994,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644171+0000","last_change":"2024-09-16T11:16:47.485172+0000","last_active":"2024-09-16T11:16:57.644171+0000","last_peered":"2024-09-16T11:16:57.644171+0000","last_clean":"2024-09-16T11:16:57.644171+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:16:57.644171+0000","last_undegraded":"2024-09-16T11:16:57.644171+0000","last_fullsized":"2024-09-16T11:16:57.644171+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:10.056185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968679+0000","last_change":"2024-09-16T11:16:47.480778+0000","last_active":"2024-09-16T11:16:57.968679+0000","last_peered":"2024-09-16T11:16:57.968679+0000","last_clean":"2024-09-16T11:16:57.968679+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:16:57.968679+0000","last_undegraded":"2024-09-16T11:16:57.968679+0000","last_fullsized":"2024-09-16T11:16:57.968679+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:29.212441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"205'491","reported_seq":8704,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968268+0000","last_change":"2024-09-16T11:16:47.486651+0000","last_active":"2024-09-16T11:16:57.968268+0000","last_peered":"2024-09-16T11:16:57.968268+0000","last_clean":"2024-09-16T11:16:57.968268+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:16:57.968268+0000","last_undegraded":"2024-09-16T11:16:57.968268+0000","last_fullsized":"2024-09-16T11:16:57.968268+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:45.637509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647126+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:16:57.647126+0000","last_peered":"2024-09-16T11:16:57.647126+0000","last_clean":"2024-09-16T11:16:57.647126+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:16:57.647126+0000","last_undegraded":"2024-09-16T11:16:57.647126+0000","last_fullsized":"2024-09-16T11:16:57.647126+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:53:16.972857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648781+0000","last_change":"2024-09-16T11:16:43.567954+0000","last_active":"2024-09-16T11:16:57.648781+0000","last_peered":"2024-09-16T11:16:57.648781+0000","last_clean":"2024-09-16T11:16:57.648781+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:16:57.648781+0000","last_undegraded":"2024-09-16T11:16:57.648781+0000","last_fullsized":"2024-09-16T11:16:57.648781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:22:11.421317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"206'644","reported_seq":6808,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.968521+0000","last_change":"2024-09-16T11:16:47.483421+0000","last_active":"2024-09-16T11:16:57.968521+0000","last_peered":"2024-09-16T11:16:57.968521+0000","last_clean":"2024-09-16T11:16:57.968521+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:16:57.968521+0000","last_undegraded":"2024-09-16T11:16:57.968521+0000","last_fullsized":"2024-09-16T11:16:57.968521+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:55:13.778395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":168,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.967896+0000","last_change":"2024-09-16T11:16:47.481364+0000","last_active":"2024-09-16T11:16:57.967896+0000","last_peered":"2024-09-16T11:16:57.967896+0000","last_clean":"2024-09-16T11:16:57.967896+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:16:57.967896+0000","last_undegraded":"2024-09-16T11:16:57.967896+0000","last_fullsized":"2024-09-16T11:16:57.967896+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:20:22.268248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969784+0000","last_change":"2024-09-16T11:14:20.062578+0000","last_active":"2024-09-16T11:16:57.969784+0000","last_peered":"2024-09-16T11:16:57.969784+0000","last_clean":"2024-09-16T11:16:57.969784+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:16:57.969784+0000","last_undegraded":"2024-09-16T11:16:57.969784+0000","last_fullsized":"2024-09-16T11:16:57.969784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:48:31.110593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"193'684","reported_seq":5002,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969111+0000","last_change":"2024-09-16T11:16:50.477765+0000","last_active":"2024-09-16T11:16:57.969111+0000","last_peered":"2024-09-16T11:16:57.969111+0000","last_clean":"2024-09-16T11:16:57.969111+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:16:57.969111+0000","last_undegraded":"2024-09-16T11:16:57.969111+0000","last_fullsized":"2024-09-16T11:16:57.969111+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:14.471034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"212'2","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644047+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:16:57.644047+0000","last_peered":"2024-09-16T11:16:57.644047+0000","last_clean":"2024-09-16T11:16:57.644047+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:16:57.644047+0000","last_undegraded":"2024-09-16T11:16:57.644047+0000","last_fullsized":"2024-09-16T11:16:57.644047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:42.036338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969075+0000","last_change":"2024-09-16T11:16:42.521626+0000","last_active":"2024-09-16T11:16:57.969075+0000","last_peered":"2024-09-16T11:16:57.969075+0000","last_clean":"2024-09-16T11:16:57.969075+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:16:57.969075+0000","last_undegraded":"2024-09-16T11:16:57.969075+0000","last_fullsized":"2024-09-16T11:16:57.969075+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:03.898472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"207'538","reported_seq":4282,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644466+0000","last_change":"2024-09-16T11:16:36.479810+0000","last_active":"2024-09-16T11:16:57.644466+0000","last_peered":"2024-09-16T11:16:57.644466+0000","last_clean":"2024-09-16T11:16:57.644466+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:16:57.644466+0000","last_undegraded":"2024-09-16T11:16:57.644466+0000","last_fullsized":"2024-09-16T11:16:57.644466+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:23:09.533692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"168'4","reported_seq":289,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969684+0000","last_change":"2024-09-16T11:16:43.545938+0000","last_active":"2024-09-16T11:16:57.969684+0000","last_peered":"2024-09-16T11:16:57.969684+0000","last_clean":"2024-09-16T11:16:57.969684+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:16:57.969684+0000","last_undegraded":"2024-09-16T11:16:57.969684+0000","last_fullsized":"2024-09-16T11:16:57.969684+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:11:35.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648992+0000","last_change":"2024-09-16T11:16:43.565231+0000","last_active":"2024-09-16T11:16:57.648992+0000","last_peered":"2024-09-16T11:16:57.648992+0000","last_clean":"2024-09-16T11:16:57.648992+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:16:57.648992+0000","last_undegraded":"2024-09-16T11:16:57.648992+0000","last_fullsized":"2024-09-16T11:16:57.648992+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:45.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":30,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970000+0000","last_change":"2024-09-16T11:16:42.529324+0000","last_active":"2024-09-16T11:16:57.970000+0000","last_peered":"2024-09-16T11:16:57.970000+0000","last_clean":"2024-09-16T11:16:57.970000+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:16:57.970000+0000","last_undegraded":"2024-09-16T11:16:57.970000+0000","last_fullsized":"2024-09-16T11:16:57.970000+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:09.000275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"205'573","reported_seq":6327,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:33.633708+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"202'885","reported_seq":3505,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648499+0000","last_change":"2024-09-16T11:16:48.483747+0000","last_active":"2024-09-16T11:16:57.648499+0000","last_peered":"2024-09-16T11:16:57.648499+0000","last_clean":"2024-09-16T11:16:57.648499+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:16:57.648499+0000","last_undegraded":"2024-09-16T11:16:57.648499+0000","last_fullsized":"2024-09-16T11:16:57.648499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11038,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646703+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:57.646703+0000","last_peered":"2024-09-16T11:16:57.646703+0000","last_clean":"2024-09-16T11:16:57.646703+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:57.646703+0000","last_undegraded":"2024-09-16T11:16:57.646703+0000","last_fullsized":"2024-09-16T11:16:57.646703+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:08.661865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"205'573","reported_seq":5145,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:37.481009+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:45.260823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"212'668","reported_seq":8071,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.644042+0000","last_change":"2024-09-16T11:16:39.501093+0000","last_active":"2024-09-16T11:16:57.644042+0000","last_peered":"2024-09-16T11:16:57.644042+0000","last_clean":"2024-09-16T11:16:57.644042+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:16:57.644042+0000","last_undegraded":"2024-09-16T11:16:57.644042+0000","last_fullsized":"2024-09-16T11:16:57.644042+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:41:46.228140+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"206'1205","reported_seq":10338,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.646826+0000","last_change":"2024-09-16T11:16:33.632197+0000","last_active":"2024-09-16T11:16:57.646826+0000","last_peered":"2024-09-16T11:16:57.646826+0000","last_clean":"2024-09-16T11:16:57.646826+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:16:57.646826+0000","last_undegraded":"2024-09-16T11:16:57.646826+0000","last_fullsized":"2024-09-16T11:16:57.646826+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:44:00.052662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17294,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647063+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:57.647063+0000","last_peered":"2024-09-16T11:16:57.647063+0000","last_clean":"2024-09-16T11:16:57.647063+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:57.647063+0000","last_undegraded":"2024-09-16T11:16:57.647063+0000","last_fullsized":"2024-09-16T11:16:57.647063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:22:53.663088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"200'769","reported_seq":8815,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.647188+0000","last_change":"2024-09-16T11:16:33.635099+0000","last_active":"2024-09-16T11:16:57.647188+0000","last_peered":"2024-09-16T11:16:57.647188+0000","last_clean":"2024-09-16T11:16:57.647188+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:16:57.647188+0000","last_undegraded":"2024-09-16T11:16:57.647188+0000","last_fullsized":"2024-09-16T11:16:57.647188+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:33.812931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"200'540","reported_seq":5263,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969308+0000","last_change":"2024-09-16T11:16:47.488023+0000","last_active":"2024-09-16T11:16:57.969308+0000","last_peered":"2024-09-16T11:16:57.969308+0000","last_clean":"2024-09-16T11:16:57.969308+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:16:57.969308+0000","last_undegraded":"2024-09-16T11:16:57.969308+0000","last_fullsized":"2024-09-16T11:16:57.969308+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:30:54.721072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23473,"ondisk_log_size":23473,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034055,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305468,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066372,"statfs":{"total":96636764160,"available":96323964928,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300092,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071748,"statfs":{"total":96636764160,"available":96329469952,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.0600000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":1.296},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":0.93899999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.999}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":1.1060000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.2390000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.403},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":1.462},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":1.448}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.755}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.879}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.085},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.157}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.002}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.2070000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.052},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.286}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312580,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059260,"statfs":{"total":96636764160,"available":96316682240,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.36399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.139,"5min":1.329,"15min":1.329},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.1379999999999999,"5min":1.173,"15min":1.173},"last":1.1259999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.86699999999999999,"15min":0.90100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.29,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.75600000000000001,"15min":0.78200000000000003},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95999999999999996,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.874,"15min":0.91800000000000004},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.026,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0029999999999999,"15min":1.0469999999999999},"min":{"1min":0.48699999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.304,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.278}]},{"osd":3,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.85999999999999999,"15min":0.875},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0649999999999999,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.98799999999999999,"15min":1.038},"min":{"1min":0.58299999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.1120000000000001,"5min":2.077,"15min":2.077},"last":1.3129999999999999}]},{"osd":5,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.02,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.2829999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.417},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.91500000000000004,"15min":0.94999999999999996},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0980000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.25}]},{"osd":6,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.069,"15min":1.1299999999999999},"min":{"1min":0.52100000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.256,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.119,"15min":1.1419999999999999},"min":{"1min":0.627,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.355,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.508}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.456},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.6399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.503},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.64600000000000002}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2785280,"data_stored":1427036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83199999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.083},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":2.0720000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":2.1520000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.498},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":2.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308412,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063428,"statfs":{"total":96636764160,"available":96320950272,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.84099999999999997}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.89900000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":1.0980000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.212},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":1.371}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":1.0029999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.4530000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":1.4039999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.5529999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313148,"kb_used_data":3304,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058692,"statfs":{"total":96636764160,"available":96316100608,"internally_reserved":0,"allocated":3383296,"data_stored":2017487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.0780000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":1.036}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.27},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":1.3160000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.22},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.8400000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.538}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:03.676 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:17:03.973 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:03.973 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":231,"fsid":"7e3ff0c6-8090-4b11-95d6-fd611813de49","created":"2024-09-16T11:12:35.459871+0000","modified":"2024-09-16T11:16:58.650867+0000","last_up_change":"2024-09-16T11:16:40.476259+0000","last_in_change":"2024-09-16T11:16:42.499018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:12:48.004741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:12:53.361644+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-09-16T11:13:02.524586+0000","flags":8201,"flags_names":"hashpspool,incomplete_clones,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"26","last_force_op_resend_prenautilus":"26","last_force_op_resend_preluminous":"26","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":24,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":2,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-09-16T11:13:11.876887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":87,"snap_epoch":206,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b772f030-6c52-46c6-a992-36cadd7f9edd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":22248},{"type":"v1","addr":"172.21.15.162:6825","nonce":22248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":22248},{"type":"v1","addr":"172.21.15.162:6827","nonce":22248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":22248},{"type":"v1","addr":"172.21.15.162:6831","nonce":22248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":22248},{"type":"v1","addr":"172.21.15.162:6829","nonce":22248}]},"public_addr":"172.21.15.162:6825/22248","cluster_addr":"172.21.15.162:6827/22248","heartbeat_back_addr":"172.21.15.162:6831/22248","heartbeat_front_addr":"172.21.15.162:6829/22248","state":["exists","up"]},{"osd":1,"uuid":"6335aa8f-a82b-45a4-be04-b1b064314f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":224,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":22247},{"type":"v1","addr":"172.21.15.162:6801","nonce":22247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":22247},{"type":"v1","addr":"172.21.15.162:6803","nonce":22247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":22247},{"type":"v1","addr":"172.21.15.162:6807","nonce":22247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":22247},{"type":"v1","addr":"172.21.15.162:6805","nonce":22247}]},"public_addr":"172.21.15.162:6801/22247","cluster_addr":"172.21.15.162:6803/22247","heartbeat_back_addr":"172.21.15.162:6807/22247","heartbeat_front_addr":"172.21.15.162:6805/22247","state":["exists","up"]},{"osd":2,"uuid":"8d9432da-1713-44e1-9c96-db33f21cf333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":22258},{"type":"v1","addr":"172.21.15.162:6817","nonce":22258}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":22258},{"type":"v1","addr":"172.21.15.162:6819","nonce":22258}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":22258},{"type":"v1","addr":"172.21.15.162:6823","nonce":22258}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":22258},{"type":"v1","addr":"172.21.15.162:6821","nonce":22258}]},"public_addr":"172.21.15.162:6817/22258","cluster_addr":"172.21.15.162:6819/22258","heartbeat_back_addr":"172.21.15.162:6823/22258","heartbeat_front_addr":"172.21.15.162:6821/22258","state":["exists","up"]},{"osd":3,"uuid":"f17d5292-6b43-4887-a146-a647c60e4f86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":225,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":22257},{"type":"v1","addr":"172.21.15.162:6809","nonce":22257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":22257},{"type":"v1","addr":"172.21.15.162:6811","nonce":22257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":22257},{"type":"v1","addr":"172.21.15.162:6815","nonce":22257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":22257},{"type":"v1","addr":"172.21.15.162:6813","nonce":22257}]},"public_addr":"172.21.15.162:6809/22257","cluster_addr":"172.21.15.162:6811/22257","heartbeat_back_addr":"172.21.15.162:6815/22257","heartbeat_front_addr":"172.21.15.162:6813/22257","state":["exists","up"]},{"osd":4,"uuid":"93cb4050-ba1d-44c4-8e05-854a4cbb9236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":22379},{"type":"v1","addr":"172.21.15.165:6801","nonce":22379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":22379},{"type":"v1","addr":"172.21.15.165:6803","nonce":22379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":22379},{"type":"v1","addr":"172.21.15.165:6807","nonce":22379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":22379},{"type":"v1","addr":"172.21.15.165:6805","nonce":22379}]},"public_addr":"172.21.15.165:6801/22379","cluster_addr":"172.21.15.165:6803/22379","heartbeat_back_addr":"172.21.15.165:6807/22379","heartbeat_front_addr":"172.21.15.165:6805/22379","state":["exists","up"]},{"osd":5,"uuid":"432a0446-4f3b-436d-bba0-76184260e13c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":220,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":22380},{"type":"v1","addr":"172.21.15.165:6809","nonce":22380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":22380},{"type":"v1","addr":"172.21.15.165:6811","nonce":22380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":22380},{"type":"v1","addr":"172.21.15.165:6815","nonce":22380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":22380},{"type":"v1","addr":"172.21.15.165:6813","nonce":22380}]},"public_addr":"172.21.15.165:6809/22380","cluster_addr":"172.21.15.165:6811/22380","heartbeat_back_addr":"172.21.15.165:6815/22380","heartbeat_front_addr":"172.21.15.165:6813/22380","state":["exists","up"]},{"osd":6,"uuid":"7c7e65e2-ab06-47f4-8d6d-712aa308e13f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":22381},{"type":"v1","addr":"172.21.15.165:6818","nonce":22381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6819","nonce":22381},{"type":"v1","addr":"172.21.15.165:6821","nonce":22381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":22381},{"type":"v1","addr":"172.21.15.165:6827","nonce":22381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":22381},{"type":"v1","addr":"172.21.15.165:6824","nonce":22381}]},"public_addr":"172.21.15.165:6818/22381","cluster_addr":"172.21.15.165:6821/22381","heartbeat_back_addr":"172.21.15.165:6827/22381","heartbeat_front_addr":"172.21.15.165:6824/22381","state":["exists","up"]},{"osd":7,"uuid":"827daf60-a01e-40a5-9595-7ec5406e4a8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":181,"up_from":213,"up_thru":218,"down_at":182,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6817","nonce":32265},{"type":"v1","addr":"172.21.15.165:6820","nonce":32265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6823","nonce":32265},{"type":"v1","addr":"172.21.15.165:6825","nonce":32265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":32265},{"type":"v1","addr":"172.21.15.165:6831","nonce":32265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":32265},{"type":"v1","addr":"172.21.15.165:6829","nonce":32265}]},"public_addr":"172.21.15.165:6820/32265","cluster_addr":"172.21.15.165:6825/32265","heartbeat_back_addr":"172.21.15.165:6831/32265","heartbeat_front_addr":"172.21.15.165:6829/32265","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.054827+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.474331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.753279+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.733978+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.980000+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.142568+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:42.883591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:12:43.129808+0000","dead_epoch":182}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:17:04.989 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T11:17:04.989 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:05.143 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:05.144 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:05.144 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:05.156 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T11:17:05.457 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T11:17:05.469 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T11:17:05.469 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:05.603 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:05.603 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:05.603 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:05.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T11:17:05.907 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T11:17:05.917 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T11:17:05.917 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:06.057 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:06.058 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:06.058 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:06.069 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T11:17:06.359 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T11:17:06.369 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T11:17:06.369 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:06.513 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:06.514 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:06.514 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:06.526 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T11:17:06.836 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T11:17:06.847 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T11:17:06.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:06.982 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:06.982 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:06.982 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:06.994 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T11:17:07.287 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T11:17:07.297 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T11:17:07.297 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:07.443 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:07.443 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:07.443 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:07.455 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T11:17:07.757 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T11:17:07.767 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T11:17:07.768 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:07.912 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:07.912 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:17:07.912 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:07.925 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T11:17:08.217 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T11:17:08.226 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T11:17:08.227 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:17:08.371 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-16T11:17:08.371 INFO:teuthology.orchestra.run.smithi162.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-16T11:17:08.371 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-16T11:17:08.383 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T11:17:08.678 INFO:teuthology.orchestra.run.smithi162.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T11:17:08.692 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:08.992 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:08.995 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:09.006 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-16T11:17:08.016163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23564,"ondisk_log_size":23564,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444148,"kb_used_data":23052,"kb_used_omap":48,"kb_used_meta":2420943,"kb_avail":752530572,"statfs":{"total":773094113280,"available":770591305728,"internally_reserved":0,"allocated":23605248,"data_stored":12646580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2479045869},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002915"},"pg_stats":[{"pgid":"4.1f","version":"206'498","reported_seq":7631,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969898+0000","last_change":"2024-09-16T11:16:46.480011+0000","last_active":"2024-09-16T11:16:57.969898+0000","last_peered":"2024-09-16T11:16:57.969898+0000","last_clean":"2024-09-16T11:16:57.969898+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:16:57.969898+0000","last_undegraded":"2024-09-16T11:16:57.969898+0000","last_fullsized":"2024-09-16T11:16:57.969898+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:17.579128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12721,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027340+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:59.027340+0000","last_peered":"2024-09-16T11:16:59.027340+0000","last_clean":"2024-09-16T11:16:59.027340+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:59.027340+0000","last_undegraded":"2024-09-16T11:16:59.027340+0000","last_fullsized":"2024-09-16T11:16:59.027340+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"205'510","reported_seq":5434,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025931+0000","last_change":"2024-09-16T11:16:34.642934+0000","last_active":"2024-09-16T11:16:59.025931+0000","last_peered":"2024-09-16T11:16:59.025931+0000","last_clean":"2024-09-16T11:16:59.025931+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:16:59.025931+0000","last_undegraded":"2024-09-16T11:16:59.025931+0000","last_fullsized":"2024-09-16T11:16:59.025931+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"206'1074","reported_seq":9479,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662286+0000","last_change":"2024-09-16T11:16:38.490863+0000","last_active":"2024-09-16T11:16:58.662286+0000","last_peered":"2024-09-16T11:16:58.662286+0000","last_clean":"2024-09-16T11:16:58.662286+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:16:58.662286+0000","last_undegraded":"2024-09-16T11:16:58.662286+0000","last_fullsized":"2024-09-16T11:16:58.662286+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7522,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662419+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:58.662419+0000","last_peered":"2024-09-16T11:16:58.662419+0000","last_clean":"2024-09-16T11:16:58.662419+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:58.662419+0000","last_undegraded":"2024-09-16T11:16:58.662419+0000","last_fullsized":"2024-09-16T11:16:58.662419+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"206'701","reported_seq":8135,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662277+0000","last_change":"2024-09-16T11:16:33.634941+0000","last_active":"2024-09-16T11:16:58.662277+0000","last_peered":"2024-09-16T11:16:58.662277+0000","last_clean":"2024-09-16T11:16:58.662277+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:16:58.662277+0000","last_undegraded":"2024-09-16T11:16:58.662277+0000","last_fullsized":"2024-09-16T11:16:58.662277+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"199'840","reported_seq":3435,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025519+0000","last_change":"2024-09-16T11:16:33.627164+0000","last_active":"2024-09-16T11:16:59.025519+0000","last_peered":"2024-09-16T11:16:59.025519+0000","last_clean":"2024-09-16T11:16:59.025519+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:16:59.025519+0000","last_undegraded":"2024-09-16T11:16:59.025519+0000","last_fullsized":"2024-09-16T11:16:59.025519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"206'775","reported_seq":5544,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662124+0000","last_change":"2024-09-16T11:16:33.633666+0000","last_active":"2024-09-16T11:16:58.662124+0000","last_peered":"2024-09-16T11:16:58.662124+0000","last_clean":"2024-09-16T11:16:58.662124+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:16:58.662124+0000","last_undegraded":"2024-09-16T11:16:58.662124+0000","last_fullsized":"2024-09-16T11:16:58.662124+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7724,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026457+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:59.026457+0000","last_peered":"2024-09-16T11:16:59.026457+0000","last_clean":"2024-09-16T11:16:59.026457+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:59.026457+0000","last_undegraded":"2024-09-16T11:16:59.026457+0000","last_fullsized":"2024-09-16T11:16:59.026457+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"200'499","reported_seq":7621,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027423+0000","last_change":"2024-09-16T11:16:33.624591+0000","last_active":"2024-09-16T11:16:59.027423+0000","last_peered":"2024-09-16T11:16:59.027423+0000","last_clean":"2024-09-16T11:16:59.027423+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:16:59.027423+0000","last_undegraded":"2024-09-16T11:16:59.027423+0000","last_fullsized":"2024-09-16T11:16:59.027423+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"205'486","reported_seq":4883,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648892+0000","last_change":"2024-09-16T11:16:45.480011+0000","last_active":"2024-09-16T11:16:57.648892+0000","last_peered":"2024-09-16T11:16:57.648892+0000","last_clean":"2024-09-16T11:16:57.648892+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:16:57.648892+0000","last_undegraded":"2024-09-16T11:16:57.648892+0000","last_fullsized":"2024-09-16T11:16:57.648892+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:43:23.227245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"205'673","reported_seq":7417,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026306+0000","last_change":"2024-09-16T11:16:39.492405+0000","last_active":"2024-09-16T11:16:59.026306+0000","last_peered":"2024-09-16T11:16:59.026306+0000","last_clean":"2024-09-16T11:16:59.026306+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:16:59.026306+0000","last_undegraded":"2024-09-16T11:16:59.026306+0000","last_fullsized":"2024-09-16T11:16:59.026306+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"205'593","reported_seq":4531,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662668+0000","last_change":"2024-09-16T11:16:33.633503+0000","last_active":"2024-09-16T11:16:58.662668+0000","last_peered":"2024-09-16T11:16:58.662668+0000","last_clean":"2024-09-16T11:16:58.662668+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:16:58.662668+0000","last_undegraded":"2024-09-16T11:16:58.662668+0000","last_fullsized":"2024-09-16T11:16:58.662668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"206'531","reported_seq":6109,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026266+0000","last_change":"2024-09-16T11:16:47.482430+0000","last_active":"2024-09-16T11:16:59.026266+0000","last_peered":"2024-09-16T11:16:59.026266+0000","last_clean":"2024-09-16T11:16:59.026266+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:16:59.026266+0000","last_undegraded":"2024-09-16T11:16:59.026266+0000","last_fullsized":"2024-09-16T11:16:59.026266+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8208,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025472+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:59.025472+0000","last_peered":"2024-09-16T11:16:59.025472+0000","last_clean":"2024-09-16T11:16:59.025472+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:59.025472+0000","last_undegraded":"2024-09-16T11:16:59.025472+0000","last_fullsized":"2024-09-16T11:16:59.025472+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"193'627","reported_seq":7680,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026035+0000","last_change":"2024-09-16T11:16:35.483110+0000","last_active":"2024-09-16T11:16:59.026035+0000","last_peered":"2024-09-16T11:16:59.026035+0000","last_clean":"2024-09-16T11:16:59.026035+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:16:59.026035+0000","last_undegraded":"2024-09-16T11:16:59.026035+0000","last_fullsized":"2024-09-16T11:16:59.026035+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":242,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969871+0000","last_change":"2024-09-16T11:16:42.522836+0000","last_active":"2024-09-16T11:16:57.969871+0000","last_peered":"2024-09-16T11:16:57.969871+0000","last_clean":"2024-09-16T11:16:57.969871+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:16:57.969871+0000","last_undegraded":"2024-09-16T11:16:57.969871+0000","last_fullsized":"2024-09-16T11:16:57.969871+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:17:31.057363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"200'1007","reported_seq":6374,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662413+0000","last_change":"2024-09-16T11:16:46.477020+0000","last_active":"2024-09-16T11:16:58.662413+0000","last_peered":"2024-09-16T11:16:58.662413+0000","last_clean":"2024-09-16T11:16:58.662413+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:16:58.662413+0000","last_undegraded":"2024-09-16T11:16:58.662413+0000","last_fullsized":"2024-09-16T11:16:58.662413+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":104,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026585+0000","last_change":"2024-09-16T11:16:47.479923+0000","last_active":"2024-09-16T11:16:59.026585+0000","last_peered":"2024-09-16T11:16:59.026585+0000","last_clean":"2024-09-16T11:16:59.026585+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:16:59.026585+0000","last_undegraded":"2024-09-16T11:16:59.026585+0000","last_fullsized":"2024-09-16T11:16:59.026585+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6184,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025914+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:59.025914+0000","last_peered":"2024-09-16T11:16:59.025914+0000","last_clean":"2024-09-16T11:16:59.025914+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:59.025914+0000","last_undegraded":"2024-09-16T11:16:59.025914+0000","last_fullsized":"2024-09-16T11:16:59.025914+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969274+0000","last_change":"2024-09-16T11:16:42.520140+0000","last_active":"2024-09-16T11:16:57.969274+0000","last_peered":"2024-09-16T11:16:57.969274+0000","last_clean":"2024-09-16T11:16:57.969274+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:16:57.969274+0000","last_undegraded":"2024-09-16T11:16:57.969274+0000","last_fullsized":"2024-09-16T11:16:57.969274+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:55:01.926666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"207'1204","reported_seq":18995,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027288+0000","last_change":"2024-09-16T11:16:47.485172+0000","last_active":"2024-09-16T11:16:59.027288+0000","last_peered":"2024-09-16T11:16:59.027288+0000","last_clean":"2024-09-16T11:16:59.027288+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:16:59.027288+0000","last_undegraded":"2024-09-16T11:16:59.027288+0000","last_fullsized":"2024-09-16T11:16:59.027288+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":189,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026536+0000","last_change":"2024-09-16T11:16:47.480778+0000","last_active":"2024-09-16T11:16:59.026536+0000","last_peered":"2024-09-16T11:16:59.026536+0000","last_clean":"2024-09-16T11:16:59.026536+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:16:59.026536+0000","last_undegraded":"2024-09-16T11:16:59.026536+0000","last_fullsized":"2024-09-16T11:16:59.026536+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"231'501","reported_seq":8725,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:06.395129+0000","last_change":"2024-09-16T11:17:06.395097+0000","last_active":"2024-09-16T11:17:06.395129+0000","last_peered":"2024-09-16T11:17:06.395129+0000","last_clean":"2024-09-16T11:17:06.395129+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:17:06.395129+0000","last_undegraded":"2024-09-16T11:17:06.395129+0000","last_fullsized":"2024-09-16T11:17:06.395129+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'501","last_scrub_stamp":"2024-09-16T11:17:06.394983+0000","last_deep_scrub":"231'501","last_deep_scrub_stamp":"2024-09-16T11:17:06.394983+0000","last_clean_scrub_stamp":"2024-09-16T11:17:06.394983+0000","objects_scrubbed":10,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:54.880464+0000","scrub_duration":0.014661676,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":125,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662413+0000","last_change":"2024-09-16T11:15:01.686934+0000","last_active":"2024-09-16T11:16:58.662413+0000","last_peered":"2024-09-16T11:16:58.662413+0000","last_clean":"2024-09-16T11:16:58.662413+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:16:58.662413+0000","last_undegraded":"2024-09-16T11:16:58.662413+0000","last_fullsized":"2024-09-16T11:16:58.662413+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648781+0000","last_change":"2024-09-16T11:16:43.567954+0000","last_active":"2024-09-16T11:16:57.648781+0000","last_peered":"2024-09-16T11:16:57.648781+0000","last_clean":"2024-09-16T11:16:57.648781+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:16:57.648781+0000","last_undegraded":"2024-09-16T11:16:57.648781+0000","last_fullsized":"2024-09-16T11:16:57.648781+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:22:11.421317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"206'644","reported_seq":6809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026166+0000","last_change":"2024-09-16T11:16:47.483421+0000","last_active":"2024-09-16T11:16:59.026166+0000","last_peered":"2024-09-16T11:16:59.026166+0000","last_clean":"2024-09-16T11:16:59.026166+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:16:59.026166+0000","last_undegraded":"2024-09-16T11:16:59.026166+0000","last_fullsized":"2024-09-16T11:16:59.026166+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":179,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:07.379372+0000","last_change":"2024-09-16T11:17:07.379321+0000","last_active":"2024-09-16T11:17:07.379372+0000","last_peered":"2024-09-16T11:17:07.379372+0000","last_clean":"2024-09-16T11:17:07.379372+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:17:07.379372+0000","last_undegraded":"2024-09-16T11:17:07.379372+0000","last_fullsized":"2024-09-16T11:17:07.379372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:07.379127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:07.379127+0000","last_clean_scrub_stamp":"2024-09-16T11:17:07.379127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:20:52.146571+0000","scrub_duration":0.002888856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:06.575074+0000","last_change":"2024-09-16T11:17:06.575020+0000","last_active":"2024-09-16T11:17:06.575074+0000","last_peered":"2024-09-16T11:17:06.575074+0000","last_clean":"2024-09-16T11:17:06.575074+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:17:06.575074+0000","last_undegraded":"2024-09-16T11:17:06.575074+0000","last_fullsized":"2024-09-16T11:17:06.575074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:06.574799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:06.574799+0000","last_clean_scrub_stamp":"2024-09-16T11:17:06.574799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:47.316896+0000","scrub_duration":0.0031164629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"193'684","reported_seq":5003,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026725+0000","last_change":"2024-09-16T11:16:50.477765+0000","last_active":"2024-09-16T11:16:59.026725+0000","last_peered":"2024-09-16T11:16:59.026725+0000","last_clean":"2024-09-16T11:16:59.026725+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:16:59.026725+0000","last_undegraded":"2024-09-16T11:16:59.026725+0000","last_fullsized":"2024-09-16T11:16:59.026725+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"212'2","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027235+0000","last_change":"2024-09-16T11:15:01.690400+0000","last_active":"2024-09-16T11:16:59.027235+0000","last_peered":"2024-09-16T11:16:59.027235+0000","last_clean":"2024-09-16T11:16:59.027235+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:16:59.027235+0000","last_undegraded":"2024-09-16T11:16:59.027235+0000","last_fullsized":"2024-09-16T11:16:59.027235+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969075+0000","last_change":"2024-09-16T11:16:42.521626+0000","last_active":"2024-09-16T11:16:57.969075+0000","last_peered":"2024-09-16T11:16:57.969075+0000","last_clean":"2024-09-16T11:16:57.969075+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:16:57.969075+0000","last_undegraded":"2024-09-16T11:16:57.969075+0000","last_fullsized":"2024-09-16T11:16:57.969075+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:03.898472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"207'538","reported_seq":4283,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027497+0000","last_change":"2024-09-16T11:16:36.479810+0000","last_active":"2024-09-16T11:16:59.027497+0000","last_peered":"2024-09-16T11:16:59.027497+0000","last_clean":"2024-09-16T11:16:59.027497+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:16:59.027497+0000","last_undegraded":"2024-09-16T11:16:59.027497+0000","last_fullsized":"2024-09-16T11:16:59.027497+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"168'4","reported_seq":290,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025981+0000","last_change":"2024-09-16T11:16:43.545938+0000","last_active":"2024-09-16T11:16:59.025981+0000","last_peered":"2024-09-16T11:16:59.025981+0000","last_clean":"2024-09-16T11:16:59.025981+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:16:59.025981+0000","last_undegraded":"2024-09-16T11:16:59.025981+0000","last_fullsized":"2024-09-16T11:16:59.025981+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:03.007348+0000","last_clean_scrub_stamp":"2024-09-16T11:13:03.007348+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":27,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648992+0000","last_change":"2024-09-16T11:16:43.565231+0000","last_active":"2024-09-16T11:16:57.648992+0000","last_peered":"2024-09-16T11:16:57.648992+0000","last_clean":"2024-09-16T11:16:57.648992+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:16:57.648992+0000","last_undegraded":"2024-09-16T11:16:57.648992+0000","last_fullsized":"2024-09-16T11:16:57.648992+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:53.794331+0000","last_clean_scrub_stamp":"2024-09-16T11:12:53.794331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:03:45.375049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":30,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.970000+0000","last_change":"2024-09-16T11:16:42.529324+0000","last_active":"2024-09-16T11:16:57.970000+0000","last_peered":"2024-09-16T11:16:57.970000+0000","last_clean":"2024-09-16T11:16:57.970000+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:16:57.970000+0000","last_undegraded":"2024-09-16T11:16:57.970000+0000","last_fullsized":"2024-09-16T11:16:57.970000+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:12:48.754572+0000","last_clean_scrub_stamp":"2024-09-16T11:12:48.754572+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:09.000275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"205'573","reported_seq":6327,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:33.633708+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:44:09.910563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"202'885","reported_seq":3505,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.648499+0000","last_change":"2024-09-16T11:16:48.483747+0000","last_active":"2024-09-16T11:16:57.648499+0000","last_peered":"2024-09-16T11:16:57.648499+0000","last_clean":"2024-09-16T11:16:57.648499+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:16:57.648499+0000","last_undegraded":"2024-09-16T11:16:57.648499+0000","last_fullsized":"2024-09-16T11:16:57.648499+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11039,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662261+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:58.662261+0000","last_peered":"2024-09-16T11:16:58.662261+0000","last_clean":"2024-09-16T11:16:58.662261+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:58.662261+0000","last_undegraded":"2024-09-16T11:16:58.662261+0000","last_fullsized":"2024-09-16T11:16:58.662261+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"205'573","reported_seq":5145,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-16T11:16:57.969602+0000","last_change":"2024-09-16T11:16:37.481009+0000","last_active":"2024-09-16T11:16:57.969602+0000","last_peered":"2024-09-16T11:16:57.969602+0000","last_clean":"2024-09-16T11:16:57.969602+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:16:57.969602+0000","last_undegraded":"2024-09-16T11:16:57.969602+0000","last_fullsized":"2024-09-16T11:16:57.969602+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:45.260823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"212'668","reported_seq":8072,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027237+0000","last_change":"2024-09-16T11:16:39.501093+0000","last_active":"2024-09-16T11:16:59.027237+0000","last_peered":"2024-09-16T11:16:59.027237+0000","last_clean":"2024-09-16T11:16:59.027237+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:16:59.027237+0000","last_undegraded":"2024-09-16T11:16:59.027237+0000","last_fullsized":"2024-09-16T11:16:59.027237+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"206'1205","reported_seq":10339,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662597+0000","last_change":"2024-09-16T11:16:33.632197+0000","last_active":"2024-09-16T11:16:58.662597+0000","last_peered":"2024-09-16T11:16:58.662597+0000","last_clean":"2024-09-16T11:16:58.662597+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:16:58.662597+0000","last_undegraded":"2024-09-16T11:16:58.662597+0000","last_fullsized":"2024-09-16T11:16:58.662597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17295,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662682+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:58.662682+0000","last_peered":"2024-09-16T11:16:58.662682+0000","last_clean":"2024-09-16T11:16:58.662682+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:58.662682+0000","last_undegraded":"2024-09-16T11:16:58.662682+0000","last_fullsized":"2024-09-16T11:16:58.662682+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"200'769","reported_seq":8816,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662584+0000","last_change":"2024-09-16T11:16:33.635099+0000","last_active":"2024-09-16T11:16:58.662584+0000","last_peered":"2024-09-16T11:16:58.662584+0000","last_clean":"2024-09-16T11:16:58.662584+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:16:58.662584+0000","last_undegraded":"2024-09-16T11:16:58.662584+0000","last_fullsized":"2024-09-16T11:16:58.662584+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"200'540","reported_seq":5264,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026725+0000","last_change":"2024-09-16T11:16:47.488023+0000","last_active":"2024-09-16T11:16:59.026725+0000","last_peered":"2024-09-16T11:16:59.026725+0000","last_clean":"2024-09-16T11:16:59.026725+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:16:59.026725+0000","last_undegraded":"2024-09-16T11:16:59.026725+0000","last_fullsized":"2024-09-16T11:16:59.026725+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23483,"ondisk_log_size":23483,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034056,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305468,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066372,"statfs":{"total":96636764160,"available":96323964928,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]}]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300092,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071748,"statfs":{"total":96636764160,"available":96329469952,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.83699999999999997}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":0.749}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.87}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":0.628}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.03}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.091},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.079}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":0.96299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300096,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071744,"statfs":{"total":96636764160,"available":96329465856,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":1.0289999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.179}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.129},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.93600000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.2330000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.411}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.3}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.151}]}]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312580,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059260,"statfs":{"total":96636764160,"available":96316682240,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73399999999999999,"15min":0.755},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0920000000000001,"5min":1.329,"15min":1.329},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81699999999999995,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.78100000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.89500000000000002},"min":{"1min":0.48799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.751,"15min":0.78000000000000003},"min":{"1min":0.54700000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.748}]},{"osd":2,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84599999999999997,"15min":0.90900000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1679999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98199999999999998,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.278,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.85199999999999998,"15min":0.872},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1990000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.97999999999999998,"15min":1.036},"min":{"1min":0.66200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.099}]},{"osd":5,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.99099999999999999,"15min":1.0649999999999999},"min":{"1min":0.499,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.25,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.028,"15min":1.1160000000000001},"min":{"1min":0.51100000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3740000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.66100000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.508,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.032}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.431}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300104,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071736,"statfs":{"total":96636764160,"available":96329457664,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.73399999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.83299999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.0069999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.77900000000000003}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.97599999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.876},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":0.95299999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.131}]}]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304248,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067592,"statfs":{"total":96636764160,"available":96325214208,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.53200000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.94099999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.85599999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.014},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":1.125}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.179}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308408,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063432,"statfs":{"total":96636764160,"available":96320954368,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.47199999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.67400000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":0.75}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":0.83299999999999996}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.70299999999999996}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":0.79600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313152,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058688,"statfs":{"total":96636764160,"available":96316096512,"internally_reserved":0,"allocated":3395584,"data_stored":2023677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.624},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.93200000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":1.05}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.089}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.2010000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.3799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.013 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.014 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T11:13:03.007348+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.015 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T11:13:03.007348+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T11:13:03.007348+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T11:12:53.794331+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T11:12:48.754572+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=12, tm_sec=48, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.016 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:09.017 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:17:29.019 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:29.317 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:29.320 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:29.332 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-16T11:17:28.022772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590659,"num_objects":304,"num_object_clones":0,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28589,"num_write_kb":754221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23791,"ondisk_log_size":23791,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452428,"kb_used_data":23052,"kb_used_omap":48,"kb_used_meta":2429263,"kb_avail":752522292,"statfs":{"total":773094113280,"available":770582827008,"internally_reserved":0,"allocated":23605248,"data_stored":12646580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2487565549},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003934"},"pg_stats":[{"pgid":"4.1f","version":"231'507","reported_seq":7651,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:25.665190+0000","last_change":"2024-09-16T11:17:25.665132+0000","last_active":"2024-09-16T11:17:25.665190+0000","last_peered":"2024-09-16T11:17:25.665190+0000","last_clean":"2024-09-16T11:17:25.665190+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:17:25.665190+0000","last_undegraded":"2024-09-16T11:17:25.665190+0000","last_fullsized":"2024-09-16T11:17:25.665190+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'507","last_scrub_stamp":"2024-09-16T11:17:25.665000+0000","last_deep_scrub":"231'507","last_deep_scrub_stamp":"2024-09-16T11:17:25.665000+0000","last_clean_scrub_stamp":"2024-09-16T11:17:25.665000+0000","objects_scrubbed":9,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:36.968099+0000","scrub_duration":0.015725313000000001,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"193'866","reported_seq":12721,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.027340+0000","last_change":"2024-09-16T11:16:46.479624+0000","last_active":"2024-09-16T11:16:59.027340+0000","last_peered":"2024-09-16T11:16:59.027340+0000","last_clean":"2024-09-16T11:16:59.027340+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:16:59.027340+0000","last_undegraded":"2024-09-16T11:16:59.027340+0000","last_fullsized":"2024-09-16T11:16:59.027340+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"231'518","reported_seq":5452,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:16.812050+0000","last_change":"2024-09-16T11:17:16.812003+0000","last_active":"2024-09-16T11:17:16.812050+0000","last_peered":"2024-09-16T11:17:16.812050+0000","last_clean":"2024-09-16T11:17:16.812050+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:17:16.812050+0000","last_undegraded":"2024-09-16T11:17:16.812050+0000","last_fullsized":"2024-09-16T11:17:16.812050+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'518","last_scrub_stamp":"2024-09-16T11:17:16.811837+0000","last_deep_scrub":"231'518","last_deep_scrub_stamp":"2024-09-16T11:17:16.811837+0000","last_clean_scrub_stamp":"2024-09-16T11:17:16.811837+0000","objects_scrubbed":8,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:31.385567+0000","scrub_duration":0.016274493000000001,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"231'1087","reported_seq":9502,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:08.578711+0000","last_change":"2024-09-16T11:17:08.578666+0000","last_active":"2024-09-16T11:17:08.578711+0000","last_peered":"2024-09-16T11:17:08.578711+0000","last_clean":"2024-09-16T11:17:08.578711+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:17:08.578711+0000","last_undegraded":"2024-09-16T11:17:08.578711+0000","last_fullsized":"2024-09-16T11:17:08.578711+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1087","last_scrub_stamp":"2024-09-16T11:17:08.578548+0000","last_deep_scrub":"231'1087","last_deep_scrub_stamp":"2024-09-16T11:17:08.578548+0000","last_clean_scrub_stamp":"2024-09-16T11:17:08.578548+0000","objects_scrubbed":13,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:35.300211+0000","scrub_duration":0.022809457000000002,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"206'517","reported_seq":7522,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662419+0000","last_change":"2024-09-16T11:16:33.632122+0000","last_active":"2024-09-16T11:16:58.662419+0000","last_peered":"2024-09-16T11:16:58.662419+0000","last_clean":"2024-09-16T11:16:58.662419+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:16:58.662419+0000","last_undegraded":"2024-09-16T11:16:58.662419+0000","last_fullsized":"2024-09-16T11:16:58.662419+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"231'708","reported_seq":8152,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:27.301958+0000","last_change":"2024-09-16T11:17:27.301851+0000","last_active":"2024-09-16T11:17:27.301958+0000","last_peered":"2024-09-16T11:17:27.301958+0000","last_clean":"2024-09-16T11:17:27.301958+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:17:27.301958+0000","last_undegraded":"2024-09-16T11:17:27.301958+0000","last_fullsized":"2024-09-16T11:17:27.301958+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'708","last_scrub_stamp":"2024-09-16T11:17:27.301590+0000","last_deep_scrub":"231'708","last_deep_scrub_stamp":"2024-09-16T11:17:27.301590+0000","last_clean_scrub_stamp":"2024-09-16T11:17:27.301590+0000","objects_scrubbed":7,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:45.289092+0000","scrub_duration":0.015344968,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"231'849","reported_seq":3454,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.829147+0000","last_change":"2024-09-16T11:17:14.829100+0000","last_active":"2024-09-16T11:17:14.829147+0000","last_peered":"2024-09-16T11:17:14.829147+0000","last_clean":"2024-09-16T11:17:14.829147+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:17:14.829147+0000","last_undegraded":"2024-09-16T11:17:14.829147+0000","last_fullsized":"2024-09-16T11:17:14.829147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'849","last_scrub_stamp":"2024-09-16T11:17:14.828953+0000","last_deep_scrub":"231'849","last_deep_scrub_stamp":"2024-09-16T11:17:14.828953+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.828953+0000","objects_scrubbed":9,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:55.952791+0000","scrub_duration":0.018398709999999999,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"231'784","reported_seq":5563,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:20.254102+0000","last_change":"2024-09-16T11:17:20.254021+0000","last_active":"2024-09-16T11:17:20.254102+0000","last_peered":"2024-09-16T11:17:20.254102+0000","last_clean":"2024-09-16T11:17:20.254102+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:17:20.254102+0000","last_undegraded":"2024-09-16T11:17:20.254102+0000","last_fullsized":"2024-09-16T11:17:20.254102+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'784","last_scrub_stamp":"2024-09-16T11:17:20.253843+0000","last_deep_scrub":"231'784","last_deep_scrub_stamp":"2024-09-16T11:17:20.253843+0000","last_clean_scrub_stamp":"2024-09-16T11:17:20.253843+0000","objects_scrubbed":9,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:14.184068+0000","scrub_duration":0.017595616000000001,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"208'1572","reported_seq":7724,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.026457+0000","last_change":"2024-09-16T11:16:53.601833+0000","last_active":"2024-09-16T11:16:59.026457+0000","last_peered":"2024-09-16T11:16:59.026457+0000","last_clean":"2024-09-16T11:16:59.026457+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:16:59.026457+0000","last_undegraded":"2024-09-16T11:16:59.026457+0000","last_fullsized":"2024-09-16T11:16:59.026457+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1572,"log_dups_size":0,"ondisk_log_size":1572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"231'511","reported_seq":7643,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:24.739990+0000","last_change":"2024-09-16T11:17:24.739961+0000","last_active":"2024-09-16T11:17:24.739990+0000","last_peered":"2024-09-16T11:17:24.739990+0000","last_clean":"2024-09-16T11:17:24.739990+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:17:24.739990+0000","last_undegraded":"2024-09-16T11:17:24.739990+0000","last_fullsized":"2024-09-16T11:17:24.739990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'511","last_scrub_stamp":"2024-09-16T11:17:24.739861+0000","last_deep_scrub":"231'511","last_deep_scrub_stamp":"2024-09-16T11:17:24.739861+0000","last_clean_scrub_stamp":"2024-09-16T11:17:24.739861+0000","objects_scrubbed":12,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:34:43.243865+0000","scrub_duration":0.021966432000000001,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"231'501","reported_seq":4909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.979607+0000","last_change":"2024-09-16T11:17:14.979573+0000","last_active":"2024-09-16T11:17:14.979607+0000","last_peered":"2024-09-16T11:17:14.979607+0000","last_clean":"2024-09-16T11:17:14.979607+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:17:14.979607+0000","last_undegraded":"2024-09-16T11:17:14.979607+0000","last_fullsized":"2024-09-16T11:17:14.979607+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'501","last_scrub_stamp":"2024-09-16T11:17:14.979468+0000","last_deep_scrub":"231'501","last_deep_scrub_stamp":"2024-09-16T11:17:14.979468+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.979468+0000","objects_scrubbed":15,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:57.772191+0000","scrub_duration":0.021363509999999999,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"231'682","reported_seq":7436,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:15.835273+0000","last_change":"2024-09-16T11:17:15.835217+0000","last_active":"2024-09-16T11:17:15.835273+0000","last_peered":"2024-09-16T11:17:15.835273+0000","last_clean":"2024-09-16T11:17:15.835273+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:17:15.835273+0000","last_undegraded":"2024-09-16T11:17:15.835273+0000","last_fullsized":"2024-09-16T11:17:15.835273+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'682","last_scrub_stamp":"2024-09-16T11:17:15.835086+0000","last_deep_scrub":"231'682","last_deep_scrub_stamp":"2024-09-16T11:17:15.835086+0000","last_clean_scrub_stamp":"2024-09-16T11:17:15.835086+0000","objects_scrubbed":9,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:43.286773+0000","scrub_duration":0.017773323000000001,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"231'601","reported_seq":4549,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:11.333885+0000","last_change":"2024-09-16T11:17:11.333853+0000","last_active":"2024-09-16T11:17:11.333885+0000","last_peered":"2024-09-16T11:17:11.333885+0000","last_clean":"2024-09-16T11:17:11.333885+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:17:11.333885+0000","last_undegraded":"2024-09-16T11:17:11.333885+0000","last_fullsized":"2024-09-16T11:17:11.333885+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'601","last_scrub_stamp":"2024-09-16T11:17:11.333742+0000","last_deep_scrub":"231'601","last_deep_scrub_stamp":"2024-09-16T11:17:11.333742+0000","last_clean_scrub_stamp":"2024-09-16T11:17:11.333742+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:20:47.314778+0000","scrub_duration":0.014920545,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"231'536","reported_seq":6124,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:10.401505+0000","last_change":"2024-09-16T11:17:10.401466+0000","last_active":"2024-09-16T11:17:10.401505+0000","last_peered":"2024-09-16T11:17:10.401505+0000","last_clean":"2024-09-16T11:17:10.401505+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:17:10.401505+0000","last_undegraded":"2024-09-16T11:17:10.401505+0000","last_fullsized":"2024-09-16T11:17:10.401505+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'536","last_scrub_stamp":"2024-09-16T11:17:10.401338+0000","last_deep_scrub":"231'536","last_deep_scrub_stamp":"2024-09-16T11:17:10.401338+0000","last_clean_scrub_stamp":"2024-09-16T11:17:10.401338+0000","objects_scrubbed":5,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:48.673871+0000","scrub_duration":0.013377053,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"207'750","reported_seq":8208,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025472+0000","last_change":"2024-09-16T11:16:47.483153+0000","last_active":"2024-09-16T11:16:59.025472+0000","last_peered":"2024-09-16T11:16:59.025472+0000","last_clean":"2024-09-16T11:16:59.025472+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:16:59.025472+0000","last_undegraded":"2024-09-16T11:16:59.025472+0000","last_fullsized":"2024-09-16T11:16:59.025472+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"231'637","reported_seq":7700,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:25.921524+0000","last_change":"2024-09-16T11:17:25.921491+0000","last_active":"2024-09-16T11:17:25.921524+0000","last_peered":"2024-09-16T11:17:25.921524+0000","last_clean":"2024-09-16T11:17:25.921524+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:17:25.921524+0000","last_undegraded":"2024-09-16T11:17:25.921524+0000","last_fullsized":"2024-09-16T11:17:25.921524+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'637","last_scrub_stamp":"2024-09-16T11:17:25.921372+0000","last_deep_scrub":"231'637","last_deep_scrub_stamp":"2024-09-16T11:17:25.921372+0000","last_clean_scrub_stamp":"2024-09-16T11:17:25.921372+0000","objects_scrubbed":10,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:01.376875+0000","scrub_duration":0.017236120000000001,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:13.630297+0000","last_change":"2024-09-16T11:17:13.630231+0000","last_active":"2024-09-16T11:17:13.630297+0000","last_peered":"2024-09-16T11:17:13.630297+0000","last_clean":"2024-09-16T11:17:13.630297+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:17:13.630297+0000","last_undegraded":"2024-09-16T11:17:13.630297+0000","last_fullsized":"2024-09-16T11:17:13.630297+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:13.629993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:13.629993+0000","last_clean_scrub_stamp":"2024-09-16T11:17:13.629993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:04.125005+0000","scrub_duration":0.0033614199999999999,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"231'1022","reported_seq":6399,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:15.390353+0000","last_change":"2024-09-16T11:17:15.390308+0000","last_active":"2024-09-16T11:17:15.390353+0000","last_peered":"2024-09-16T11:17:15.390353+0000","last_clean":"2024-09-16T11:17:15.390353+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:17:15.390353+0000","last_undegraded":"2024-09-16T11:17:15.390353+0000","last_fullsized":"2024-09-16T11:17:15.390353+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1022","last_scrub_stamp":"2024-09-16T11:17:15.390180+0000","last_deep_scrub":"231'1022","last_deep_scrub_stamp":"2024-09-16T11:17:15.390180+0000","last_clean_scrub_stamp":"2024-09-16T11:17:15.390180+0000","objects_scrubbed":15,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:45.993658+0000","scrub_duration":0.02410344,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:16.397988+0000","last_change":"2024-09-16T11:17:16.397917+0000","last_active":"2024-09-16T11:17:16.397988+0000","last_peered":"2024-09-16T11:17:16.397988+0000","last_clean":"2024-09-16T11:17:16.397988+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:17:16.397988+0000","last_undegraded":"2024-09-16T11:17:16.397988+0000","last_fullsized":"2024-09-16T11:17:16.397988+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:16.397699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:16.397699+0000","last_clean_scrub_stamp":"2024-09-16T11:17:16.397699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:55.941965+0000","scrub_duration":0.003387279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"202'429","reported_seq":6184,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:59.025914+0000","last_change":"2024-09-16T11:16:53.601698+0000","last_active":"2024-09-16T11:16:59.025914+0000","last_peered":"2024-09-16T11:16:59.025914+0000","last_clean":"2024-09-16T11:16:59.025914+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:16:59.025914+0000","last_undegraded":"2024-09-16T11:16:59.025914+0000","last_fullsized":"2024-09-16T11:16:59.025914+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:11.253236+0000","last_change":"2024-09-16T11:17:11.253164+0000","last_active":"2024-09-16T11:17:11.253236+0000","last_peered":"2024-09-16T11:17:11.253236+0000","last_clean":"2024-09-16T11:17:11.253236+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:17:11.253236+0000","last_undegraded":"2024-09-16T11:17:11.253236+0000","last_fullsized":"2024-09-16T11:17:11.253236+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:11.252931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:11.252931+0000","last_clean_scrub_stamp":"2024-09-16T11:17:11.252931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:44.312001+0000","scrub_duration":0.0040079429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"231'1213","reported_seq":19014,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:21.655004+0000","last_change":"2024-09-16T11:17:21.654976+0000","last_active":"2024-09-16T11:17:21.655004+0000","last_peered":"2024-09-16T11:17:21.655004+0000","last_clean":"2024-09-16T11:17:21.655004+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:17:21.655004+0000","last_undegraded":"2024-09-16T11:17:21.655004+0000","last_fullsized":"2024-09-16T11:17:21.655004+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1213","last_scrub_stamp":"2024-09-16T11:17:21.654864+0000","last_deep_scrub":"231'1213","last_deep_scrub_stamp":"2024-09-16T11:17:21.654864+0000","last_clean_scrub_stamp":"2024-09-16T11:17:21.654864+0000","objects_scrubbed":9,"log_size":1213,"log_dups_size":0,"ondisk_log_size":1213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:51.958147+0000","scrub_duration":0.021150305000000001,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":199,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:17.433411+0000","last_change":"2024-09-16T11:17:17.433344+0000","last_active":"2024-09-16T11:17:17.433411+0000","last_peered":"2024-09-16T11:17:17.433411+0000","last_clean":"2024-09-16T11:17:17.433411+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:17:17.433411+0000","last_undegraded":"2024-09-16T11:17:17.433411+0000","last_fullsized":"2024-09-16T11:17:17.433411+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:17.433118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:17.433118+0000","last_clean_scrub_stamp":"2024-09-16T11:17:17.433118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:33.963096+0000","scrub_duration":0.0034588760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"231'501","reported_seq":8725,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:06.395129+0000","last_change":"2024-09-16T11:17:06.395097+0000","last_active":"2024-09-16T11:17:06.395129+0000","last_peered":"2024-09-16T11:17:06.395129+0000","last_clean":"2024-09-16T11:17:06.395129+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:17:06.395129+0000","last_undegraded":"2024-09-16T11:17:06.395129+0000","last_fullsized":"2024-09-16T11:17:06.395129+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'501","last_scrub_stamp":"2024-09-16T11:17:06.394983+0000","last_deep_scrub":"231'501","last_deep_scrub_stamp":"2024-09-16T11:17:06.394983+0000","last_clean_scrub_stamp":"2024-09-16T11:17:06.394983+0000","objects_scrubbed":10,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:54.880464+0000","scrub_duration":0.014661676,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:13.384277+0000","last_change":"2024-09-16T11:17:13.384187+0000","last_active":"2024-09-16T11:17:13.384277+0000","last_peered":"2024-09-16T11:17:13.384277+0000","last_clean":"2024-09-16T11:17:13.384277+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:17:13.384277+0000","last_undegraded":"2024-09-16T11:17:13.384277+0000","last_fullsized":"2024-09-16T11:17:13.384277+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:13.383943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:13.383943+0000","last_clean_scrub_stamp":"2024-09-16T11:17:13.383943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:38:20.334991+0000","scrub_duration":0.0038229200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"231'3","reported_seq":39,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:12.966898+0000","last_change":"2024-09-16T11:17:12.966828+0000","last_active":"2024-09-16T11:17:12.966898+0000","last_peered":"2024-09-16T11:17:12.966898+0000","last_clean":"2024-09-16T11:17:12.966898+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:17:12.966898+0000","last_undegraded":"2024-09-16T11:17:12.966898+0000","last_fullsized":"2024-09-16T11:17:12.966898+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'3","last_scrub_stamp":"2024-09-16T11:17:12.966598+0000","last_deep_scrub":"231'3","last_deep_scrub_stamp":"2024-09-16T11:17:12.966598+0000","last_clean_scrub_stamp":"2024-09-16T11:17:12.966598+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:30.311207+0000","scrub_duration":0.0064621160000000004,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"231'649","reported_seq":6824,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.372303+0000","last_change":"2024-09-16T11:17:14.372261+0000","last_active":"2024-09-16T11:17:14.372303+0000","last_peered":"2024-09-16T11:17:14.372303+0000","last_clean":"2024-09-16T11:17:14.372303+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:17:14.372303+0000","last_undegraded":"2024-09-16T11:17:14.372303+0000","last_fullsized":"2024-09-16T11:17:14.372303+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'649","last_scrub_stamp":"2024-09-16T11:17:14.372149+0000","last_deep_scrub":"231'649","last_deep_scrub_stamp":"2024-09-16T11:17:14.372149+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.372149+0000","objects_scrubbed":5,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:17.974164+0000","scrub_duration":0.012108511000000001,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":179,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:07.379372+0000","last_change":"2024-09-16T11:17:07.379321+0000","last_active":"2024-09-16T11:17:07.379372+0000","last_peered":"2024-09-16T11:17:07.379372+0000","last_clean":"2024-09-16T11:17:07.379372+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:17:07.379372+0000","last_undegraded":"2024-09-16T11:17:07.379372+0000","last_fullsized":"2024-09-16T11:17:07.379372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:07.379127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:07.379127+0000","last_clean_scrub_stamp":"2024-09-16T11:17:07.379127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:20:52.146571+0000","scrub_duration":0.002888856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:06.575074+0000","last_change":"2024-09-16T11:17:06.575020+0000","last_active":"2024-09-16T11:17:06.575074+0000","last_peered":"2024-09-16T11:17:06.575074+0000","last_clean":"2024-09-16T11:17:06.575074+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:17:06.575074+0000","last_undegraded":"2024-09-16T11:17:06.575074+0000","last_fullsized":"2024-09-16T11:17:06.575074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:06.574799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:06.574799+0000","last_clean_scrub_stamp":"2024-09-16T11:17:06.574799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:47.316896+0000","scrub_duration":0.0031164629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"231'687","reported_seq":5016,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:19.471825+0000","last_change":"2024-09-16T11:17:19.471719+0000","last_active":"2024-09-16T11:17:19.471825+0000","last_peered":"2024-09-16T11:17:19.471825+0000","last_clean":"2024-09-16T11:17:19.471825+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:17:19.471825+0000","last_undegraded":"2024-09-16T11:17:19.471825+0000","last_fullsized":"2024-09-16T11:17:19.471825+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'687","last_scrub_stamp":"2024-09-16T11:17:19.471501+0000","last_deep_scrub":"231'687","last_deep_scrub_stamp":"2024-09-16T11:17:19.471501+0000","last_clean_scrub_stamp":"2024-09-16T11:17:19.471501+0000","objects_scrubbed":3,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:21.889836+0000","scrub_duration":0.0093935359999999992,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"231'4","reported_seq":255,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:16.590463+0000","last_change":"2024-09-16T11:17:16.590388+0000","last_active":"2024-09-16T11:17:16.590463+0000","last_peered":"2024-09-16T11:17:16.590463+0000","last_clean":"2024-09-16T11:17:16.590463+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:17:16.590463+0000","last_undegraded":"2024-09-16T11:17:16.590463+0000","last_fullsized":"2024-09-16T11:17:16.590463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'4","last_scrub_stamp":"2024-09-16T11:17:16.590172+0000","last_deep_scrub":"231'4","last_deep_scrub_stamp":"2024-09-16T11:17:16.590172+0000","last_clean_scrub_stamp":"2024-09-16T11:17:16.590172+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:51.848942+0000","scrub_duration":0.0076028689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:18.235542+0000","last_change":"2024-09-16T11:17:18.235411+0000","last_active":"2024-09-16T11:17:18.235542+0000","last_peered":"2024-09-16T11:17:18.235542+0000","last_clean":"2024-09-16T11:17:18.235542+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:17:18.235542+0000","last_undegraded":"2024-09-16T11:17:18.235542+0000","last_fullsized":"2024-09-16T11:17:18.235542+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:18.235158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:18.235158+0000","last_clean_scrub_stamp":"2024-09-16T11:17:18.235158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:45.086072+0000","scrub_duration":0.0036654180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"231'548","reported_seq":4303,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:19.636709+0000","last_change":"2024-09-16T11:17:19.636603+0000","last_active":"2024-09-16T11:17:19.636709+0000","last_peered":"2024-09-16T11:17:19.636709+0000","last_clean":"2024-09-16T11:17:19.636709+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:17:19.636709+0000","last_undegraded":"2024-09-16T11:17:19.636709+0000","last_fullsized":"2024-09-16T11:17:19.636709+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'548","last_scrub_stamp":"2024-09-16T11:17:19.636202+0000","last_deep_scrub":"231'548","last_deep_scrub_stamp":"2024-09-16T11:17:19.636202+0000","last_clean_scrub_stamp":"2024-09-16T11:17:19.636202+0000","objects_scrubbed":10,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:41:36.169466+0000","scrub_duration":0.017388913999999998,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"231'8","reported_seq":304,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:10.898872+0000","last_change":"2024-09-16T11:17:10.898809+0000","last_active":"2024-09-16T11:17:10.898872+0000","last_peered":"2024-09-16T11:17:10.898872+0000","last_clean":"2024-09-16T11:17:10.898872+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:17:10.898872+0000","last_undegraded":"2024-09-16T11:17:10.898872+0000","last_fullsized":"2024-09-16T11:17:10.898872+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'8","last_scrub_stamp":"2024-09-16T11:17:10.898586+0000","last_deep_scrub":"231'8","last_deep_scrub_stamp":"2024-09-16T11:17:10.898586+0000","last_clean_scrub_stamp":"2024-09-16T11:17:10.898586+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:03.215088+0000","scrub_duration":0.010267887,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"231'2","reported_seq":39,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:09.013208+0000","last_change":"2024-09-16T11:17:09.013146+0000","last_active":"2024-09-16T11:17:09.013208+0000","last_peered":"2024-09-16T11:17:09.013208+0000","last_clean":"2024-09-16T11:17:09.013208+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:17:09.013208+0000","last_undegraded":"2024-09-16T11:17:09.013208+0000","last_fullsized":"2024-09-16T11:17:09.013208+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'2","last_scrub_stamp":"2024-09-16T11:17:09.012898+0000","last_deep_scrub":"231'2","last_deep_scrub_stamp":"2024-09-16T11:17:09.012898+0000","last_clean_scrub_stamp":"2024-09-16T11:17:09.012898+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:56.275593+0000","scrub_duration":0.0063723850000000004,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"231'74","reported_seq":43,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.324804+0000","last_change":"2024-09-16T11:17:14.324762+0000","last_active":"2024-09-16T11:17:14.324804+0000","last_peered":"2024-09-16T11:17:14.324804+0000","last_clean":"2024-09-16T11:17:14.324804+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:17:14.324804+0000","last_undegraded":"2024-09-16T11:17:14.324804+0000","last_fullsized":"2024-09-16T11:17:14.324804+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'74","last_scrub_stamp":"2024-09-16T11:17:14.324624+0000","last_deep_scrub":"231'74","last_deep_scrub_stamp":"2024-09-16T11:17:14.324624+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.324624+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:48.289862+0000","scrub_duration":0.011197525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"231'576","reported_seq":6341,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:22.640882+0000","last_change":"2024-09-16T11:17:22.640799+0000","last_active":"2024-09-16T11:17:22.640882+0000","last_peered":"2024-09-16T11:17:22.640882+0000","last_clean":"2024-09-16T11:17:22.640882+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:17:22.640882+0000","last_undegraded":"2024-09-16T11:17:22.640882+0000","last_fullsized":"2024-09-16T11:17:22.640882+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'576","last_scrub_stamp":"2024-09-16T11:17:22.640531+0000","last_deep_scrub":"231'576","last_deep_scrub_stamp":"2024-09-16T11:17:22.640531+0000","last_clean_scrub_stamp":"2024-09-16T11:17:22.640531+0000","objects_scrubbed":3,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:06.135248+0000","scrub_duration":0.010927955,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"231'893","reported_seq":3524,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:24.062791+0000","last_change":"2024-09-16T11:17:24.062709+0000","last_active":"2024-09-16T11:17:24.062791+0000","last_peered":"2024-09-16T11:17:24.062791+0000","last_clean":"2024-09-16T11:17:24.062791+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:17:24.062791+0000","last_undegraded":"2024-09-16T11:17:24.062791+0000","last_fullsized":"2024-09-16T11:17:24.062791+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'893","last_scrub_stamp":"2024-09-16T11:17:24.062553+0000","last_deep_scrub":"231'893","last_deep_scrub_stamp":"2024-09-16T11:17:24.062553+0000","last_clean_scrub_stamp":"2024-09-16T11:17:24.062553+0000","objects_scrubbed":8,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:01.104463+0000","scrub_duration":0.016182775,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"202'742","reported_seq":11039,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662261+0000","last_change":"2024-09-16T11:16:34.645856+0000","last_active":"2024-09-16T11:16:58.662261+0000","last_peered":"2024-09-16T11:16:58.662261+0000","last_clean":"2024-09-16T11:16:58.662261+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:16:58.662261+0000","last_undegraded":"2024-09-16T11:16:58.662261+0000","last_fullsized":"2024-09-16T11:16:58.662261+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"231'581","reported_seq":5164,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:15.588967+0000","last_change":"2024-09-16T11:17:15.588852+0000","last_active":"2024-09-16T11:17:15.588967+0000","last_peered":"2024-09-16T11:17:15.588967+0000","last_clean":"2024-09-16T11:17:15.588967+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:17:15.588967+0000","last_undegraded":"2024-09-16T11:17:15.588967+0000","last_fullsized":"2024-09-16T11:17:15.588967+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'581","last_scrub_stamp":"2024-09-16T11:17:15.588581+0000","last_deep_scrub":"231'581","last_deep_scrub_stamp":"2024-09-16T11:17:15.588581+0000","last_clean_scrub_stamp":"2024-09-16T11:17:15.588581+0000","objects_scrubbed":8,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:37.456199+0000","scrub_duration":0.017158735000000001,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"231'682","reported_seq":8096,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:25.704967+0000","last_change":"2024-09-16T11:17:25.704935+0000","last_active":"2024-09-16T11:17:25.704967+0000","last_peered":"2024-09-16T11:17:25.704967+0000","last_clean":"2024-09-16T11:17:25.704967+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:17:25.704967+0000","last_undegraded":"2024-09-16T11:17:25.704967+0000","last_fullsized":"2024-09-16T11:17:25.704967+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'682","last_scrub_stamp":"2024-09-16T11:17:25.704824+0000","last_deep_scrub":"231'682","last_deep_scrub_stamp":"2024-09-16T11:17:25.704824+0000","last_clean_scrub_stamp":"2024-09-16T11:17:25.704824+0000","objects_scrubbed":14,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:25.457976+0000","scrub_duration":0.021165478000000001,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"231'1216","reported_seq":10360,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.426230+0000","last_change":"2024-09-16T11:17:14.426073+0000","last_active":"2024-09-16T11:17:14.426230+0000","last_peered":"2024-09-16T11:17:14.426230+0000","last_clean":"2024-09-16T11:17:14.426230+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:17:14.426230+0000","last_undegraded":"2024-09-16T11:17:14.426230+0000","last_fullsized":"2024-09-16T11:17:14.426230+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1216","last_scrub_stamp":"2024-09-16T11:17:14.425792+0000","last_deep_scrub":"231'1216","last_deep_scrub_stamp":"2024-09-16T11:17:14.425792+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.425792+0000","objects_scrubbed":11,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:48.991766+0000","scrub_duration":0.020939989999999999,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"206'1009","reported_seq":17295,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:16:58.662682+0000","last_change":"2024-09-16T11:16:33.636485+0000","last_active":"2024-09-16T11:16:58.662682+0000","last_peered":"2024-09-16T11:16:58.662682+0000","last_clean":"2024-09-16T11:16:58.662682+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:16:58.662682+0000","last_undegraded":"2024-09-16T11:16:58.662682+0000","last_fullsized":"2024-09-16T11:16:58.662682+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:13:12.078339+0000","last_clean_scrub_stamp":"2024-09-16T11:13:12.078339+0000","objects_scrubbed":0,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"231'773","reported_seq":8830,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:17.327946+0000","last_change":"2024-09-16T11:17:17.327918+0000","last_active":"2024-09-16T11:17:17.327946+0000","last_peered":"2024-09-16T11:17:17.327946+0000","last_clean":"2024-09-16T11:17:17.327946+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:17:17.327946+0000","last_undegraded":"2024-09-16T11:17:17.327946+0000","last_fullsized":"2024-09-16T11:17:17.327946+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'773","last_scrub_stamp":"2024-09-16T11:17:17.327818+0000","last_deep_scrub":"231'773","last_deep_scrub_stamp":"2024-09-16T11:17:17.327818+0000","last_clean_scrub_stamp":"2024-09-16T11:17:17.327818+0000","objects_scrubbed":4,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:30.892162+0000","scrub_duration":0.012298748,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"231'553","reported_seq":5287,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:18.458925+0000","last_change":"2024-09-16T11:17:18.458883+0000","last_active":"2024-09-16T11:17:18.458925+0000","last_peered":"2024-09-16T11:17:18.458925+0000","last_clean":"2024-09-16T11:17:18.458925+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:17:18.458925+0000","last_undegraded":"2024-09-16T11:17:18.458925+0000","last_fullsized":"2024-09-16T11:17:18.458925+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'553","last_scrub_stamp":"2024-09-16T11:17:18.458738+0000","last_deep_scrub":"231'553","last_deep_scrub_stamp":"2024-09-16T11:17:18.458738+0000","last_clean_scrub_stamp":"2024-09-16T11:17:18.458738+0000","objects_scrubbed":13,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:18.750700+0000","scrub_duration":0.019018033,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23700,"ondisk_log_size":23700,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034060,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305460,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":302106,"kb_avail":94066380,"statfs":{"total":96636764160,"available":96323973120,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":309357445},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.2310000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":1.143}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.95299999999999996}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":1.095},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":0.82099999999999995}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":0.877},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.276}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.1859999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:15:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0920000000000001},"min":{"1min":0.67200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.4570000000000001,"5min":1.7589999999999999,"15min":1.7589999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0700000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.3220000000000001,"5min":1.661,"15min":1.661},"last":1.321}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300096,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071744,"statfs":{"total":96636764160,"available":96329465856,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.69699999999999995}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.79600000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.92100000000000004}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.131}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.1919999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.0449999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:15:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0780000000000001,"15min":1.0760000000000001},"min":{"1min":0.60499999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1080000000000001,"15min":1.1359999999999999},"min":{"1min":0.72199999999999998,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.464,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312572,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059268,"statfs":{"total":96636764160,"available":96316690432,"internally_reserved":0,"allocated":2805760,"data_stored":1433328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73399999999999999,"15min":0.755},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0920000000000001,"5min":1.329,"15min":1.329},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81699999999999995,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.0680000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.89500000000000002},"min":{"1min":0.48799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.751,"15min":0.78000000000000003},"min":{"1min":0.54700000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84599999999999997,"15min":0.90900000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1679999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.878},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98199999999999998,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.278,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0069999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.85199999999999998,"15min":0.872},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1990000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.97999999999999998,"15min":1.036},"min":{"1min":0.66200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.97799999999999998}]},{"osd":5,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.99099999999999999,"15min":1.0649999999999999},"min":{"1min":0.499,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.25,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.89900000000000002}]},{"osd":6,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.028,"15min":1.1160000000000001},"min":{"1min":0.51100000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3740000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.105},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.66100000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.508,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.141}]},{"osd":7,"last update":"Mon Sep 16 11:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0600000000000001,"15min":1.1240000000000001},"min":{"1min":0.55300000000000005,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1499999999999999,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.046},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1919999999999999,"15min":1.2649999999999999},"min":{"1min":0.68500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3069999999999999,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.1619999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300096,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071744,"statfs":{"total":96636764160,"available":96329465856,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.88800000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.1379999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":1.01}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.044},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.1040000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.2110000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.006,"15min":0.996},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.635,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.101,"15min":1.101},"min":{"1min":0.65600000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":1.4159999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308404,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063436,"statfs":{"total":96636764160,"available":96320958464,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.624},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.79300000000000004}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.70899999999999996}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":1.075},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":0.97799999999999998}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":1.1970000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.117},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.0700000000000001,"15min":1.1120000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3340000000000001,"5min":2.5630000000000002,"15min":2.5630000000000002},"last":1.2649999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308408,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063432,"statfs":{"total":96636764160,"available":96320954368,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.68300000000000005}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":0.78300000000000003}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":0.96699999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":1.032}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.1259999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.097,"15min":1.109},"min":{"1min":0.624,"5min":0.495,"15min":0.495},"max":{"1min":1.3859999999999999,"5min":2.512,"15min":2.512},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.1100000000000001,"15min":1.1599999999999999},"min":{"1min":0.57199999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.2450000000000001,"5min":2.6600000000000001,"15min":2.6600000000000001},"last":1.077}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313148,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058692,"statfs":{"total":96636764160,"available":96316100608,"internally_reserved":0,"allocated":3395584,"data_stored":2023677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.78000000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.86099999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.048},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":0.83599999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.4399999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.2410000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:15:54 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.0409999999999999,"15min":1.0449999999999999},"min":{"1min":0.66300000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.294},{"interface":"front","average":{"1min":1.022,"5min":1.0389999999999999,"15min":1.0660000000000001},"min":{"1min":0.624,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9199999999999999,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.339}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:29.336 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.336 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.336 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.336 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.337 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.337 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.337 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-16T11:13:12.078339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=13, tm_sec=12, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=17, tm_sec=3, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:17:29.337 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:17:49.338 DEBUG:teuthology.orchestra.run.smithi162:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:17:49.641 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-16T11:17:49.642 INFO:teuthology.orchestra.run.smithi162.stderr:dumped all 2024-09-16T11:17:49.658 INFO:teuthology.orchestra.run.smithi162.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-16T11:17:48.029255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590742,"num_objects":305,"num_object_clones":0,"num_object_copies":610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":298,"num_whiteouts":293,"num_read":253594,"num_read_kb":1427253,"num_write":28590,"num_write_kb":754222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":830743680,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":336,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23859,"ondisk_log_size":23859,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456596,"kb_used_data":23060,"kb_used_omap":48,"kb_used_meta":2433423,"kb_avail":752518124,"statfs":{"total":773094113280,"available":770578558976,"internally_reserved":0,"allocated":23613440,"data_stored":12646746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49939,"internal_metadata":2491825389},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":83,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003732"},"pg_stats":[{"pgid":"4.1f","version":"231'507","reported_seq":7651,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:25.665190+0000","last_change":"2024-09-16T11:17:25.665132+0000","last_active":"2024-09-16T11:17:25.665190+0000","last_peered":"2024-09-16T11:17:25.665190+0000","last_clean":"2024-09-16T11:17:25.665190+0000","last_became_active":"2024-09-16T11:16:46.477616+0000","last_became_peered":"2024-09-16T11:16:46.477616+0000","last_unstale":"2024-09-16T11:17:25.665190+0000","last_undegraded":"2024-09-16T11:17:25.665190+0000","last_fullsized":"2024-09-16T11:17:25.665190+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'507","last_scrub_stamp":"2024-09-16T11:17:25.665000+0000","last_deep_scrub":"231'507","last_deep_scrub_stamp":"2024-09-16T11:17:25.665000+0000","last_clean_scrub_stamp":"2024-09-16T11:17:25.665000+0000","objects_scrubbed":9,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:36.968099+0000","scrub_duration":0.015725313000000001,"objects_trimmed":0,"snaptrim_duration":0.0022410780000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6756,"num_read_kb":31660,"num_write":420,"num_write_kb":11403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":638,"num_bytes_recovered":17635523,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"231'879","reported_seq":12744,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:29.769582+0000","last_change":"2024-09-16T11:17:29.769538+0000","last_active":"2024-09-16T11:17:29.769582+0000","last_peered":"2024-09-16T11:17:29.769582+0000","last_clean":"2024-09-16T11:17:29.769582+0000","last_became_active":"2024-09-16T11:16:46.477395+0000","last_became_peered":"2024-09-16T11:16:46.477395+0000","last_unstale":"2024-09-16T11:17:29.769582+0000","last_undegraded":"2024-09-16T11:17:29.769582+0000","last_fullsized":"2024-09-16T11:17:29.769582+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'879","last_scrub_stamp":"2024-09-16T11:17:29.769439+0000","last_deep_scrub":"231'879","last_deep_scrub_stamp":"2024-09-16T11:17:29.769439+0000","last_clean_scrub_stamp":"2024-09-16T11:17:29.769439+0000","objects_scrubbed":13,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:32.801896+0000","scrub_duration":0.019437855,"objects_trimmed":0,"snaptrim_duration":0.003298268,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":13098,"num_read_kb":92123,"num_write":1021,"num_write_kb":43682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":678,"num_bytes_recovered":35829516,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"231'518","reported_seq":5452,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:16.812050+0000","last_change":"2024-09-16T11:17:16.812003+0000","last_active":"2024-09-16T11:17:16.812050+0000","last_peered":"2024-09-16T11:17:16.812050+0000","last_clean":"2024-09-16T11:17:16.812050+0000","last_became_active":"2024-09-16T11:16:34.638383+0000","last_became_peered":"2024-09-16T11:16:34.638383+0000","last_unstale":"2024-09-16T11:17:16.812050+0000","last_undegraded":"2024-09-16T11:17:16.812050+0000","last_fullsized":"2024-09-16T11:17:16.812050+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'518","last_scrub_stamp":"2024-09-16T11:17:16.811837+0000","last_deep_scrub":"231'518","last_deep_scrub_stamp":"2024-09-16T11:17:16.811837+0000","last_clean_scrub_stamp":"2024-09-16T11:17:16.811837+0000","objects_scrubbed":8,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:28:31.385567+0000","scrub_duration":0.016274493000000001,"objects_trimmed":0,"snaptrim_duration":0.002083877,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4719,"num_read_kb":26108,"num_write":608,"num_write_kb":14516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":7456020,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"231'1087","reported_seq":9502,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:08.578711+0000","last_change":"2024-09-16T11:17:08.578666+0000","last_active":"2024-09-16T11:17:08.578711+0000","last_peered":"2024-09-16T11:17:08.578711+0000","last_clean":"2024-09-16T11:17:08.578711+0000","last_became_active":"2024-09-16T11:16:38.486494+0000","last_became_peered":"2024-09-16T11:16:38.486494+0000","last_unstale":"2024-09-16T11:17:08.578711+0000","last_undegraded":"2024-09-16T11:17:08.578711+0000","last_fullsized":"2024-09-16T11:17:08.578711+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1087","last_scrub_stamp":"2024-09-16T11:17:08.578548+0000","last_deep_scrub":"231'1087","last_deep_scrub_stamp":"2024-09-16T11:17:08.578548+0000","last_clean_scrub_stamp":"2024-09-16T11:17:08.578548+0000","objects_scrubbed":13,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:35.300211+0000","scrub_duration":0.022809457000000002,"objects_trimmed":0,"snaptrim_duration":0.001838126,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":11503,"num_read_kb":38573,"num_write":1466,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":12087157,"num_keys_recovered":142,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"231'526","reported_seq":7541,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:34.249421+0000","last_change":"2024-09-16T11:17:34.249335+0000","last_active":"2024-09-16T11:17:34.249421+0000","last_peered":"2024-09-16T11:17:34.249421+0000","last_clean":"2024-09-16T11:17:34.249421+0000","last_became_active":"2024-09-16T11:15:07.452416+0000","last_became_peered":"2024-09-16T11:15:07.452416+0000","last_unstale":"2024-09-16T11:17:34.249421+0000","last_undegraded":"2024-09-16T11:17:34.249421+0000","last_fullsized":"2024-09-16T11:17:34.249421+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'526","last_scrub_stamp":"2024-09-16T11:17:34.249053+0000","last_deep_scrub":"231'526","last_deep_scrub_stamp":"2024-09-16T11:17:34.249053+0000","last_clean_scrub_stamp":"2024-09-16T11:17:34.249053+0000","objects_scrubbed":9,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:26.954197+0000","scrub_duration":0.018133819999999998,"objects_trimmed":0,"snaptrim_duration":0.0031667560000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7292,"num_read_kb":24293,"num_write":634,"num_write_kb":14662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":16026626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"231'708","reported_seq":8152,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:27.301958+0000","last_change":"2024-09-16T11:17:27.301851+0000","last_active":"2024-09-16T11:17:27.301958+0000","last_peered":"2024-09-16T11:17:27.301958+0000","last_clean":"2024-09-16T11:17:27.301958+0000","last_became_active":"2024-09-16T11:15:05.437529+0000","last_became_peered":"2024-09-16T11:15:05.437529+0000","last_unstale":"2024-09-16T11:17:27.301958+0000","last_undegraded":"2024-09-16T11:17:27.301958+0000","last_fullsized":"2024-09-16T11:17:27.301958+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'708","last_scrub_stamp":"2024-09-16T11:17:27.301590+0000","last_deep_scrub":"231'708","last_deep_scrub_stamp":"2024-09-16T11:17:27.301590+0000","last_clean_scrub_stamp":"2024-09-16T11:17:27.301590+0000","objects_scrubbed":7,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:45.289092+0000","scrub_duration":0.015344968,"objects_trimmed":0,"snaptrim_duration":0.0057830219999999996,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":8561,"num_read_kb":41085,"num_write":848,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":38231591,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"231'849","reported_seq":3454,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.829147+0000","last_change":"2024-09-16T11:17:14.829100+0000","last_active":"2024-09-16T11:17:14.829147+0000","last_peered":"2024-09-16T11:17:14.829147+0000","last_clean":"2024-09-16T11:17:14.829147+0000","last_became_active":"2024-09-16T11:14:23.088233+0000","last_became_peered":"2024-09-16T11:14:23.088233+0000","last_unstale":"2024-09-16T11:17:14.829147+0000","last_undegraded":"2024-09-16T11:17:14.829147+0000","last_fullsized":"2024-09-16T11:17:14.829147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'849","last_scrub_stamp":"2024-09-16T11:17:14.828953+0000","last_deep_scrub":"231'849","last_deep_scrub_stamp":"2024-09-16T11:17:14.828953+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.828953+0000","objects_scrubbed":9,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:18:55.952791+0000","scrub_duration":0.018398709999999999,"objects_trimmed":0,"snaptrim_duration":0.001333248,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":5152,"num_read_kb":21292,"num_write":1141,"num_write_kb":18692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":9384604,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"231'784","reported_seq":5563,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:20.254102+0000","last_change":"2024-09-16T11:17:20.254021+0000","last_active":"2024-09-16T11:17:20.254102+0000","last_peered":"2024-09-16T11:17:20.254102+0000","last_clean":"2024-09-16T11:17:20.254102+0000","last_became_active":"2024-09-16T11:15:08.471655+0000","last_became_peered":"2024-09-16T11:15:08.471655+0000","last_unstale":"2024-09-16T11:17:20.254102+0000","last_undegraded":"2024-09-16T11:17:20.254102+0000","last_fullsized":"2024-09-16T11:17:20.254102+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'784","last_scrub_stamp":"2024-09-16T11:17:20.253843+0000","last_deep_scrub":"231'784","last_deep_scrub_stamp":"2024-09-16T11:17:20.253843+0000","last_clean_scrub_stamp":"2024-09-16T11:17:20.253843+0000","objects_scrubbed":9,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:51:14.184068+0000","scrub_duration":0.017595616000000001,"objects_trimmed":0,"snaptrim_duration":0.0050693429999999996,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6996,"num_read_kb":36244,"num_write":826,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":27695310,"num_keys_recovered":64,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"231'1577","reported_seq":7739,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:31.971548+0000","last_change":"2024-09-16T11:17:31.971518+0000","last_active":"2024-09-16T11:17:31.971548+0000","last_peered":"2024-09-16T11:17:31.971548+0000","last_clean":"2024-09-16T11:17:31.971548+0000","last_became_active":"2024-09-16T11:16:53.599131+0000","last_became_peered":"2024-09-16T11:16:53.599131+0000","last_unstale":"2024-09-16T11:17:31.971548+0000","last_undegraded":"2024-09-16T11:17:31.971548+0000","last_fullsized":"2024-09-16T11:17:31.971548+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1577","last_scrub_stamp":"2024-09-16T11:17:31.971437+0000","last_deep_scrub":"231'1577","last_deep_scrub_stamp":"2024-09-16T11:17:31.971437+0000","last_clean_scrub_stamp":"2024-09-16T11:17:31.971437+0000","objects_scrubbed":5,"log_size":1577,"log_dups_size":0,"ondisk_log_size":1577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:01:01.472668+0000","scrub_duration":0.013741738999999999,"objects_trimmed":2,"snaptrim_duration":0.001246643,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":11855,"num_read_kb":16561,"num_write":2646,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":654,"num_bytes_recovered":10790751,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"231'511","reported_seq":7643,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:24.739990+0000","last_change":"2024-09-16T11:17:24.739961+0000","last_active":"2024-09-16T11:17:24.739990+0000","last_peered":"2024-09-16T11:17:24.739990+0000","last_clean":"2024-09-16T11:17:24.739990+0000","last_became_active":"2024-09-16T11:13:13.115795+0000","last_became_peered":"2024-09-16T11:13:13.115795+0000","last_unstale":"2024-09-16T11:17:24.739990+0000","last_undegraded":"2024-09-16T11:17:24.739990+0000","last_fullsized":"2024-09-16T11:17:24.739990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'511","last_scrub_stamp":"2024-09-16T11:17:24.739861+0000","last_deep_scrub":"231'511","last_deep_scrub_stamp":"2024-09-16T11:17:24.739861+0000","last_clean_scrub_stamp":"2024-09-16T11:17:24.739861+0000","objects_scrubbed":12,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:34:43.243865+0000","scrub_duration":0.021966432000000001,"objects_trimmed":0,"snaptrim_duration":0.0010159240000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7899,"num_read_kb":50102,"num_write":531,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"231'501","reported_seq":4909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.979607+0000","last_change":"2024-09-16T11:17:14.979573+0000","last_active":"2024-09-16T11:17:14.979607+0000","last_peered":"2024-09-16T11:17:14.979607+0000","last_clean":"2024-09-16T11:17:14.979607+0000","last_became_active":"2024-09-16T11:16:45.476971+0000","last_became_peered":"2024-09-16T11:16:45.476971+0000","last_unstale":"2024-09-16T11:17:14.979607+0000","last_undegraded":"2024-09-16T11:17:14.979607+0000","last_fullsized":"2024-09-16T11:17:14.979607+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'501","last_scrub_stamp":"2024-09-16T11:17:14.979468+0000","last_deep_scrub":"231'501","last_deep_scrub_stamp":"2024-09-16T11:17:14.979468+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.979468+0000","objects_scrubbed":15,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:57.772191+0000","scrub_duration":0.021363509999999999,"objects_trimmed":0,"snaptrim_duration":0.0015057499999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3461,"num_read_kb":16761,"num_write":416,"num_write_kb":26553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":724,"num_bytes_recovered":17767164,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"231'682","reported_seq":7436,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:15.835273+0000","last_change":"2024-09-16T11:17:15.835217+0000","last_active":"2024-09-16T11:17:15.835273+0000","last_peered":"2024-09-16T11:17:15.835273+0000","last_clean":"2024-09-16T11:17:15.835273+0000","last_became_active":"2024-09-16T11:16:39.488967+0000","last_became_peered":"2024-09-16T11:16:39.488967+0000","last_unstale":"2024-09-16T11:17:15.835273+0000","last_undegraded":"2024-09-16T11:17:15.835273+0000","last_fullsized":"2024-09-16T11:17:15.835273+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'682","last_scrub_stamp":"2024-09-16T11:17:15.835086+0000","last_deep_scrub":"231'682","last_deep_scrub_stamp":"2024-09-16T11:17:15.835086+0000","last_clean_scrub_stamp":"2024-09-16T11:17:15.835086+0000","objects_scrubbed":9,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:43.286773+0000","scrub_duration":0.017773323000000001,"objects_trimmed":0,"snaptrim_duration":0.001606985,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7403,"num_read_kb":59980,"num_write":597,"num_write_kb":27738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":24561997,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"231'601","reported_seq":4549,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:11.333885+0000","last_change":"2024-09-16T11:17:11.333853+0000","last_active":"2024-09-16T11:17:11.333885+0000","last_peered":"2024-09-16T11:17:11.333885+0000","last_clean":"2024-09-16T11:17:11.333885+0000","last_became_active":"2024-09-16T11:15:05.439383+0000","last_became_peered":"2024-09-16T11:15:05.439383+0000","last_unstale":"2024-09-16T11:17:11.333885+0000","last_undegraded":"2024-09-16T11:17:11.333885+0000","last_fullsized":"2024-09-16T11:17:11.333885+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'601","last_scrub_stamp":"2024-09-16T11:17:11.333742+0000","last_deep_scrub":"231'601","last_deep_scrub_stamp":"2024-09-16T11:17:11.333742+0000","last_clean_scrub_stamp":"2024-09-16T11:17:11.333742+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:20:47.314778+0000","scrub_duration":0.014920545,"objects_trimmed":0,"snaptrim_duration":0.00454858,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4843,"num_read_kb":38472,"num_write":587,"num_write_kb":23407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":50628682,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"231'536","reported_seq":6124,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:10.401505+0000","last_change":"2024-09-16T11:17:10.401466+0000","last_active":"2024-09-16T11:17:10.401505+0000","last_peered":"2024-09-16T11:17:10.401505+0000","last_clean":"2024-09-16T11:17:10.401505+0000","last_became_active":"2024-09-16T11:16:47.478525+0000","last_became_peered":"2024-09-16T11:16:47.478525+0000","last_unstale":"2024-09-16T11:17:10.401505+0000","last_undegraded":"2024-09-16T11:17:10.401505+0000","last_fullsized":"2024-09-16T11:17:10.401505+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'536","last_scrub_stamp":"2024-09-16T11:17:10.401338+0000","last_deep_scrub":"231'536","last_deep_scrub_stamp":"2024-09-16T11:17:10.401338+0000","last_clean_scrub_stamp":"2024-09-16T11:17:10.401338+0000","objects_scrubbed":5,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:48.673871+0000","scrub_duration":0.013377053,"objects_trimmed":1,"snaptrim_duration":0.0048529100000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":4626,"num_read_kb":53437,"num_write":734,"num_write_kb":23985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":46974829,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"231'763","reported_seq":8231,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:29.934598+0000","last_change":"2024-09-16T11:17:29.934444+0000","last_active":"2024-09-16T11:17:29.934598+0000","last_peered":"2024-09-16T11:17:29.934598+0000","last_clean":"2024-09-16T11:17:29.934598+0000","last_became_active":"2024-09-16T11:16:47.480944+0000","last_became_peered":"2024-09-16T11:16:47.480944+0000","last_unstale":"2024-09-16T11:17:29.934598+0000","last_undegraded":"2024-09-16T11:17:29.934598+0000","last_fullsized":"2024-09-16T11:17:29.934598+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'763","last_scrub_stamp":"2024-09-16T11:17:29.934163+0000","last_deep_scrub":"231'763","last_deep_scrub_stamp":"2024-09-16T11:17:29.934163+0000","last_clean_scrub_stamp":"2024-09-16T11:17:29.934163+0000","objects_scrubbed":13,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:21.558761+0000","scrub_duration":0.021731158,"objects_trimmed":0,"snaptrim_duration":0.0015993559999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8861,"num_read_kb":97051,"num_write":756,"num_write_kb":44988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":37436884,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"231'637","reported_seq":7700,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:25.921524+0000","last_change":"2024-09-16T11:17:25.921491+0000","last_active":"2024-09-16T11:17:25.921524+0000","last_peered":"2024-09-16T11:17:25.921524+0000","last_clean":"2024-09-16T11:17:25.921524+0000","last_became_active":"2024-09-16T11:16:35.479781+0000","last_became_peered":"2024-09-16T11:16:35.479781+0000","last_unstale":"2024-09-16T11:17:25.921524+0000","last_undegraded":"2024-09-16T11:17:25.921524+0000","last_fullsized":"2024-09-16T11:17:25.921524+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'637","last_scrub_stamp":"2024-09-16T11:17:25.921372+0000","last_deep_scrub":"231'637","last_deep_scrub_stamp":"2024-09-16T11:17:25.921372+0000","last_clean_scrub_stamp":"2024-09-16T11:17:25.921372+0000","objects_scrubbed":10,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:51:01.376875+0000","scrub_duration":0.017236120000000001,"objects_trimmed":0,"snaptrim_duration":0.0015821559999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7644,"num_read_kb":37743,"num_write":775,"num_write_kb":23016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":36705080,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:13.630297+0000","last_change":"2024-09-16T11:17:13.630231+0000","last_active":"2024-09-16T11:17:13.630297+0000","last_peered":"2024-09-16T11:17:13.630297+0000","last_clean":"2024-09-16T11:17:13.630297+0000","last_became_active":"2024-09-16T11:16:42.522492+0000","last_became_peered":"2024-09-16T11:16:42.522492+0000","last_unstale":"2024-09-16T11:17:13.630297+0000","last_undegraded":"2024-09-16T11:17:13.630297+0000","last_fullsized":"2024-09-16T11:17:13.630297+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:13.629993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:13.629993+0000","last_clean_scrub_stamp":"2024-09-16T11:17:13.629993+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:19:04.125005+0000","scrub_duration":0.0033614199999999999,"objects_trimmed":0,"snaptrim_duration":0.00057881000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"231'1022","reported_seq":6399,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:15.390353+0000","last_change":"2024-09-16T11:17:15.390308+0000","last_active":"2024-09-16T11:17:15.390353+0000","last_peered":"2024-09-16T11:17:15.390353+0000","last_clean":"2024-09-16T11:17:15.390353+0000","last_became_active":"2024-09-16T11:16:46.473798+0000","last_became_peered":"2024-09-16T11:16:46.473798+0000","last_unstale":"2024-09-16T11:17:15.390353+0000","last_undegraded":"2024-09-16T11:17:15.390353+0000","last_fullsized":"2024-09-16T11:17:15.390353+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1022","last_scrub_stamp":"2024-09-16T11:17:15.390180+0000","last_deep_scrub":"231'1022","last_deep_scrub_stamp":"2024-09-16T11:17:15.390180+0000","last_clean_scrub_stamp":"2024-09-16T11:17:15.390180+0000","objects_scrubbed":15,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:45.993658+0000","scrub_duration":0.02410344,"objects_trimmed":0,"snaptrim_duration":0.0044003180000000003,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8094,"num_read_kb":18444,"num_write":1170,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":12915342,"num_keys_recovered":113,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":114,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:16.397988+0000","last_change":"2024-09-16T11:17:16.397917+0000","last_active":"2024-09-16T11:17:16.397988+0000","last_peered":"2024-09-16T11:17:16.397988+0000","last_clean":"2024-09-16T11:17:16.397988+0000","last_became_active":"2024-09-16T11:16:47.479577+0000","last_became_peered":"2024-09-16T11:16:47.479577+0000","last_unstale":"2024-09-16T11:17:16.397988+0000","last_undegraded":"2024-09-16T11:17:16.397988+0000","last_fullsized":"2024-09-16T11:17:16.397988+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:16.397699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:16.397699+0000","last_clean_scrub_stamp":"2024-09-16T11:17:16.397699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:55.941965+0000","scrub_duration":0.003387279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"231'441","reported_seq":6206,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:35.035633+0000","last_change":"2024-09-16T11:17:35.035524+0000","last_active":"2024-09-16T11:17:35.035633+0000","last_peered":"2024-09-16T11:17:35.035633+0000","last_clean":"2024-09-16T11:17:35.035633+0000","last_became_active":"2024-09-16T11:16:53.599084+0000","last_became_peered":"2024-09-16T11:16:53.599084+0000","last_unstale":"2024-09-16T11:17:35.035633+0000","last_undegraded":"2024-09-16T11:17:35.035633+0000","last_fullsized":"2024-09-16T11:17:35.035633+0000","mapping_epoch":225,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":226,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'441","last_scrub_stamp":"2024-09-16T11:17:35.035256+0000","last_deep_scrub":"231'441","last_deep_scrub_stamp":"2024-09-16T11:17:35.035256+0000","last_clean_scrub_stamp":"2024-09-16T11:17:35.035256+0000","objects_scrubbed":12,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:26:10.944709+0000","scrub_duration":0.023782536,"objects_trimmed":0,"snaptrim_duration":0.0024678600000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":4467,"num_read_kb":35925,"num_write":408,"num_write_kb":11489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":15646385,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:11.253236+0000","last_change":"2024-09-16T11:17:11.253164+0000","last_active":"2024-09-16T11:17:11.253236+0000","last_peered":"2024-09-16T11:17:11.253236+0000","last_clean":"2024-09-16T11:17:11.253236+0000","last_became_active":"2024-09-16T11:16:42.519475+0000","last_became_peered":"2024-09-16T11:16:42.519475+0000","last_unstale":"2024-09-16T11:17:11.253236+0000","last_undegraded":"2024-09-16T11:17:11.253236+0000","last_fullsized":"2024-09-16T11:17:11.253236+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:11.252931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:11.252931+0000","last_clean_scrub_stamp":"2024-09-16T11:17:11.252931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:44.312001+0000","scrub_duration":0.0040079429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"231'1213","reported_seq":19014,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:21.655004+0000","last_change":"2024-09-16T11:17:21.654976+0000","last_active":"2024-09-16T11:17:21.655004+0000","last_peered":"2024-09-16T11:17:21.655004+0000","last_clean":"2024-09-16T11:17:21.655004+0000","last_became_active":"2024-09-16T11:16:47.479530+0000","last_became_peered":"2024-09-16T11:16:47.479530+0000","last_unstale":"2024-09-16T11:17:21.655004+0000","last_undegraded":"2024-09-16T11:17:21.655004+0000","last_fullsized":"2024-09-16T11:17:21.655004+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1213","last_scrub_stamp":"2024-09-16T11:17:21.654864+0000","last_deep_scrub":"231'1213","last_deep_scrub_stamp":"2024-09-16T11:17:21.654864+0000","last_clean_scrub_stamp":"2024-09-16T11:17:21.654864+0000","objects_scrubbed":9,"log_size":1213,"log_dups_size":0,"ondisk_log_size":1213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:57:51.958147+0000","scrub_duration":0.021150305000000001,"objects_trimmed":0,"snaptrim_duration":0.0040560980000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":19281,"num_read_kb":47098,"num_write":1462,"num_write_kb":20959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":19321759,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":199,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:17.433411+0000","last_change":"2024-09-16T11:17:17.433344+0000","last_active":"2024-09-16T11:17:17.433411+0000","last_peered":"2024-09-16T11:17:17.433411+0000","last_clean":"2024-09-16T11:17:17.433411+0000","last_became_active":"2024-09-16T11:16:47.480256+0000","last_became_peered":"2024-09-16T11:16:47.480256+0000","last_unstale":"2024-09-16T11:17:17.433411+0000","last_undegraded":"2024-09-16T11:17:17.433411+0000","last_fullsized":"2024-09-16T11:17:17.433411+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:17.433118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:17.433118+0000","last_clean_scrub_stamp":"2024-09-16T11:17:17.433118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:33.963096+0000","scrub_duration":0.0034588760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"231'501","reported_seq":8725,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:06.395129+0000","last_change":"2024-09-16T11:17:06.395097+0000","last_active":"2024-09-16T11:17:06.395129+0000","last_peered":"2024-09-16T11:17:06.395129+0000","last_clean":"2024-09-16T11:17:06.395129+0000","last_became_active":"2024-09-16T11:16:47.482525+0000","last_became_peered":"2024-09-16T11:16:47.482525+0000","last_unstale":"2024-09-16T11:17:06.395129+0000","last_undegraded":"2024-09-16T11:17:06.395129+0000","last_fullsized":"2024-09-16T11:17:06.395129+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'501","last_scrub_stamp":"2024-09-16T11:17:06.394983+0000","last_deep_scrub":"231'501","last_deep_scrub_stamp":"2024-09-16T11:17:06.394983+0000","last_clean_scrub_stamp":"2024-09-16T11:17:06.394983+0000","objects_scrubbed":10,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:54.880464+0000","scrub_duration":0.014661676,"objects_trimmed":0,"snaptrim_duration":0.006674765,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":7461,"num_read_kb":29121,"num_write":572,"num_write_kb":10793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":661,"num_bytes_recovered":45391986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:13.384277+0000","last_change":"2024-09-16T11:17:13.384187+0000","last_active":"2024-09-16T11:17:13.384277+0000","last_peered":"2024-09-16T11:17:13.384277+0000","last_clean":"2024-09-16T11:17:13.384277+0000","last_became_active":"2024-09-16T11:15:01.686499+0000","last_became_peered":"2024-09-16T11:15:01.686499+0000","last_unstale":"2024-09-16T11:17:13.384277+0000","last_undegraded":"2024-09-16T11:17:13.384277+0000","last_fullsized":"2024-09-16T11:17:13.384277+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:13.383943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:13.383943+0000","last_clean_scrub_stamp":"2024-09-16T11:17:13.383943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:38:20.334991+0000","scrub_duration":0.0038229200000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"231'3","reported_seq":39,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:12.966898+0000","last_change":"2024-09-16T11:17:12.966828+0000","last_active":"2024-09-16T11:17:12.966898+0000","last_peered":"2024-09-16T11:17:12.966898+0000","last_clean":"2024-09-16T11:17:12.966898+0000","last_became_active":"2024-09-16T11:16:43.519462+0000","last_became_peered":"2024-09-16T11:16:43.519462+0000","last_unstale":"2024-09-16T11:17:12.966898+0000","last_undegraded":"2024-09-16T11:17:12.966898+0000","last_fullsized":"2024-09-16T11:17:12.966898+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'3","last_scrub_stamp":"2024-09-16T11:17:12.966598+0000","last_deep_scrub":"231'3","last_deep_scrub_stamp":"2024-09-16T11:17:12.966598+0000","last_clean_scrub_stamp":"2024-09-16T11:17:12.966598+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:30.311207+0000","scrub_duration":0.0064621160000000004,"objects_trimmed":0,"snaptrim_duration":0.00097639799999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"231'649","reported_seq":6824,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.372303+0000","last_change":"2024-09-16T11:17:14.372261+0000","last_active":"2024-09-16T11:17:14.372303+0000","last_peered":"2024-09-16T11:17:14.372303+0000","last_clean":"2024-09-16T11:17:14.372303+0000","last_became_active":"2024-09-16T11:16:47.478880+0000","last_became_peered":"2024-09-16T11:16:47.478880+0000","last_unstale":"2024-09-16T11:17:14.372303+0000","last_undegraded":"2024-09-16T11:17:14.372303+0000","last_fullsized":"2024-09-16T11:17:14.372303+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'649","last_scrub_stamp":"2024-09-16T11:17:14.372149+0000","last_deep_scrub":"231'649","last_deep_scrub_stamp":"2024-09-16T11:17:14.372149+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.372149+0000","objects_scrubbed":5,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:38:17.974164+0000","scrub_duration":0.012108511000000001,"objects_trimmed":0,"snaptrim_duration":0.0037721709999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":7355,"num_read_kb":59495,"num_write":986,"num_write_kb":29396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":36799272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":179,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:07.379372+0000","last_change":"2024-09-16T11:17:07.379321+0000","last_active":"2024-09-16T11:17:07.379372+0000","last_peered":"2024-09-16T11:17:07.379372+0000","last_clean":"2024-09-16T11:17:07.379372+0000","last_became_active":"2024-09-16T11:16:47.480783+0000","last_became_peered":"2024-09-16T11:16:47.480783+0000","last_unstale":"2024-09-16T11:17:07.379372+0000","last_undegraded":"2024-09-16T11:17:07.379372+0000","last_fullsized":"2024-09-16T11:17:07.379372+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:07.379127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:07.379127+0000","last_clean_scrub_stamp":"2024-09-16T11:17:07.379127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:20:52.146571+0000","scrub_duration":0.002888856,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":173,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:06.575074+0000","last_change":"2024-09-16T11:17:06.575020+0000","last_active":"2024-09-16T11:17:06.575074+0000","last_peered":"2024-09-16T11:17:06.575074+0000","last_clean":"2024-09-16T11:17:06.575074+0000","last_became_active":"2024-09-16T11:14:20.062258+0000","last_became_peered":"2024-09-16T11:14:20.062258+0000","last_unstale":"2024-09-16T11:17:06.575074+0000","last_undegraded":"2024-09-16T11:17:06.575074+0000","last_fullsized":"2024-09-16T11:17:06.575074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:06.574799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:06.574799+0000","last_clean_scrub_stamp":"2024-09-16T11:17:06.574799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:47.316896+0000","scrub_duration":0.0031164629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"231'687","reported_seq":5016,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:19.471825+0000","last_change":"2024-09-16T11:17:19.471719+0000","last_active":"2024-09-16T11:17:19.471825+0000","last_peered":"2024-09-16T11:17:19.471825+0000","last_clean":"2024-09-16T11:17:19.471825+0000","last_became_active":"2024-09-16T11:16:50.472991+0000","last_became_peered":"2024-09-16T11:16:50.472991+0000","last_unstale":"2024-09-16T11:17:19.471825+0000","last_undegraded":"2024-09-16T11:17:19.471825+0000","last_fullsized":"2024-09-16T11:17:19.471825+0000","mapping_epoch":222,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'687","last_scrub_stamp":"2024-09-16T11:17:19.471501+0000","last_deep_scrub":"231'687","last_deep_scrub_stamp":"2024-09-16T11:17:19.471501+0000","last_clean_scrub_stamp":"2024-09-16T11:17:19.471501+0000","objects_scrubbed":3,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:43:21.889836+0000","scrub_duration":0.0093935359999999992,"objects_trimmed":0,"snaptrim_duration":0.001637856,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":3872,"num_read_kb":9908,"num_write":771,"num_write_kb":17063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1069,"num_bytes_recovered":8545378,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"231'5","reported_seq":256,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:16.590519+0000","last_change":"2024-09-16T11:17:16.590388+0000","last_active":"2024-09-16T11:17:16.590519+0000","last_peered":"2024-09-16T11:17:16.590519+0000","last_clean":"2024-09-16T11:17:16.590519+0000","last_became_active":"2024-09-16T11:15:01.686624+0000","last_became_peered":"2024-09-16T11:15:01.686624+0000","last_unstale":"2024-09-16T11:17:16.590519+0000","last_undegraded":"2024-09-16T11:17:16.590519+0000","last_fullsized":"2024-09-16T11:17:16.590519+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'4","last_scrub_stamp":"2024-09-16T11:17:16.590172+0000","last_deep_scrub":"231'4","last_deep_scrub_stamp":"2024-09-16T11:17:16.590172+0000","last_clean_scrub_stamp":"2024-09-16T11:17:16.590172+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:51.848942+0000","scrub_duration":0.0076028689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":166,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:18.235542+0000","last_change":"2024-09-16T11:17:18.235411+0000","last_active":"2024-09-16T11:17:18.235542+0000","last_peered":"2024-09-16T11:17:18.235542+0000","last_clean":"2024-09-16T11:17:18.235542+0000","last_became_active":"2024-09-16T11:16:42.521167+0000","last_became_peered":"2024-09-16T11:16:42.521167+0000","last_unstale":"2024-09-16T11:17:18.235542+0000","last_undegraded":"2024-09-16T11:17:18.235542+0000","last_fullsized":"2024-09-16T11:17:18.235542+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:17:18.235158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:17:18.235158+0000","last_clean_scrub_stamp":"2024-09-16T11:17:18.235158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:45.086072+0000","scrub_duration":0.0036654180000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"231'548","reported_seq":4303,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:19.636709+0000","last_change":"2024-09-16T11:17:19.636603+0000","last_active":"2024-09-16T11:17:19.636709+0000","last_peered":"2024-09-16T11:17:19.636709+0000","last_clean":"2024-09-16T11:17:19.636709+0000","last_became_active":"2024-09-16T11:16:36.475268+0000","last_became_peered":"2024-09-16T11:16:36.475268+0000","last_unstale":"2024-09-16T11:17:19.636709+0000","last_undegraded":"2024-09-16T11:17:19.636709+0000","last_fullsized":"2024-09-16T11:17:19.636709+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'548","last_scrub_stamp":"2024-09-16T11:17:19.636202+0000","last_deep_scrub":"231'548","last_deep_scrub_stamp":"2024-09-16T11:17:19.636202+0000","last_clean_scrub_stamp":"2024-09-16T11:17:19.636202+0000","objects_scrubbed":10,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:41:36.169466+0000","scrub_duration":0.017388913999999998,"objects_trimmed":0,"snaptrim_duration":0.002284768,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3513,"num_read_kb":98783,"num_write":505,"num_write_kb":53243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":42695854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"231'8","reported_seq":304,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:10.898872+0000","last_change":"2024-09-16T11:17:10.898809+0000","last_active":"2024-09-16T11:17:10.898872+0000","last_peered":"2024-09-16T11:17:10.898872+0000","last_clean":"2024-09-16T11:17:10.898872+0000","last_became_active":"2024-09-16T11:16:43.525012+0000","last_became_peered":"2024-09-16T11:16:43.525012+0000","last_unstale":"2024-09-16T11:17:10.898872+0000","last_undegraded":"2024-09-16T11:17:10.898872+0000","last_fullsized":"2024-09-16T11:17:10.898872+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'8","last_scrub_stamp":"2024-09-16T11:17:10.898586+0000","last_deep_scrub":"231'8","last_deep_scrub_stamp":"2024-09-16T11:17:10.898586+0000","last_clean_scrub_stamp":"2024-09-16T11:17:10.898586+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:03.215088+0000","scrub_duration":0.010267887,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"231'2","reported_seq":39,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:09.013208+0000","last_change":"2024-09-16T11:17:09.013146+0000","last_active":"2024-09-16T11:17:09.013208+0000","last_peered":"2024-09-16T11:17:09.013208+0000","last_clean":"2024-09-16T11:17:09.013208+0000","last_became_active":"2024-09-16T11:16:43.518522+0000","last_became_peered":"2024-09-16T11:16:43.518522+0000","last_unstale":"2024-09-16T11:17:09.013208+0000","last_undegraded":"2024-09-16T11:17:09.013208+0000","last_fullsized":"2024-09-16T11:17:09.013208+0000","mapping_epoch":215,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'2","last_scrub_stamp":"2024-09-16T11:17:09.012898+0000","last_deep_scrub":"231'2","last_deep_scrub_stamp":"2024-09-16T11:17:09.012898+0000","last_clean_scrub_stamp":"2024-09-16T11:17:09.012898+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:19:56.275593+0000","scrub_duration":0.0063723850000000004,"objects_trimmed":0,"snaptrim_duration":0.00066481799999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"231'74","reported_seq":43,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.324804+0000","last_change":"2024-09-16T11:17:14.324762+0000","last_active":"2024-09-16T11:17:14.324804+0000","last_peered":"2024-09-16T11:17:14.324804+0000","last_clean":"2024-09-16T11:17:14.324804+0000","last_became_active":"2024-09-16T11:16:42.522164+0000","last_became_peered":"2024-09-16T11:16:42.522164+0000","last_unstale":"2024-09-16T11:17:14.324804+0000","last_undegraded":"2024-09-16T11:17:14.324804+0000","last_fullsized":"2024-09-16T11:17:14.324804+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'74","last_scrub_stamp":"2024-09-16T11:17:14.324624+0000","last_deep_scrub":"231'74","last_deep_scrub_stamp":"2024-09-16T11:17:14.324624+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.324624+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:48.289862+0000","scrub_duration":0.011197525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"231'576","reported_seq":6341,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:22.640882+0000","last_change":"2024-09-16T11:17:22.640799+0000","last_active":"2024-09-16T11:17:22.640882+0000","last_peered":"2024-09-16T11:17:22.640882+0000","last_clean":"2024-09-16T11:17:22.640882+0000","last_became_active":"2024-09-16T11:15:06.446029+0000","last_became_peered":"2024-09-16T11:15:06.446029+0000","last_unstale":"2024-09-16T11:17:22.640882+0000","last_undegraded":"2024-09-16T11:17:22.640882+0000","last_fullsized":"2024-09-16T11:17:22.640882+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'576","last_scrub_stamp":"2024-09-16T11:17:22.640531+0000","last_deep_scrub":"231'576","last_deep_scrub_stamp":"2024-09-16T11:17:22.640531+0000","last_clean_scrub_stamp":"2024-09-16T11:17:22.640531+0000","objects_scrubbed":3,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:06.135248+0000","scrub_duration":0.010927955,"objects_trimmed":0,"snaptrim_duration":0.0045158719999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":6247,"num_read_kb":32498,"num_write":565,"num_write_kb":15376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":24085887,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"231'893","reported_seq":3524,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:24.062791+0000","last_change":"2024-09-16T11:17:24.062709+0000","last_active":"2024-09-16T11:17:24.062791+0000","last_peered":"2024-09-16T11:17:24.062791+0000","last_clean":"2024-09-16T11:17:24.062791+0000","last_became_active":"2024-09-16T11:16:48.480115+0000","last_became_peered":"2024-09-16T11:16:48.480115+0000","last_unstale":"2024-09-16T11:17:24.062791+0000","last_undegraded":"2024-09-16T11:17:24.062791+0000","last_fullsized":"2024-09-16T11:17:24.062791+0000","mapping_epoch":220,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'893","last_scrub_stamp":"2024-09-16T11:17:24.062553+0000","last_deep_scrub":"231'893","last_deep_scrub_stamp":"2024-09-16T11:17:24.062553+0000","last_clean_scrub_stamp":"2024-09-16T11:17:24.062553+0000","objects_scrubbed":8,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:26:01.104463+0000","scrub_duration":0.016182775,"objects_trimmed":0,"snaptrim_duration":0.00097503200000000003,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3766,"num_read_kb":23423,"num_write":1164,"num_write_kb":12053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":803,"num_bytes_recovered":17037594,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"231'745","reported_seq":11052,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:32.279538+0000","last_change":"2024-09-16T11:17:32.279452+0000","last_active":"2024-09-16T11:17:32.279538+0000","last_peered":"2024-09-16T11:17:32.279538+0000","last_clean":"2024-09-16T11:17:32.279538+0000","last_became_active":"2024-09-16T11:16:34.642102+0000","last_became_peered":"2024-09-16T11:16:34.642102+0000","last_unstale":"2024-09-16T11:17:32.279538+0000","last_undegraded":"2024-09-16T11:17:32.279538+0000","last_fullsized":"2024-09-16T11:17:32.279538+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'745","last_scrub_stamp":"2024-09-16T11:17:32.279201+0000","last_deep_scrub":"231'745","last_deep_scrub_stamp":"2024-09-16T11:17:32.279201+0000","last_clean_scrub_stamp":"2024-09-16T11:17:32.279201+0000","objects_scrubbed":3,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:04.617533+0000","scrub_duration":0.010040032000000001,"objects_trimmed":0,"snaptrim_duration":0.001912539,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":10947,"num_read_kb":87123,"num_write":807,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":21562321,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"231'581","reported_seq":5164,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:15.588967+0000","last_change":"2024-09-16T11:17:15.588852+0000","last_active":"2024-09-16T11:17:15.588967+0000","last_peered":"2024-09-16T11:17:15.588967+0000","last_clean":"2024-09-16T11:17:15.588967+0000","last_became_active":"2024-09-16T11:16:37.478176+0000","last_became_peered":"2024-09-16T11:16:37.478176+0000","last_unstale":"2024-09-16T11:17:15.588967+0000","last_undegraded":"2024-09-16T11:17:15.588967+0000","last_fullsized":"2024-09-16T11:17:15.588967+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'581","last_scrub_stamp":"2024-09-16T11:17:15.588581+0000","last_deep_scrub":"231'581","last_deep_scrub_stamp":"2024-09-16T11:17:15.588581+0000","last_clean_scrub_stamp":"2024-09-16T11:17:15.588581+0000","objects_scrubbed":8,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:37.456199+0000","scrub_duration":0.017158735000000001,"objects_trimmed":0,"snaptrim_duration":0.0012723789999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":3812,"num_read_kb":45861,"num_write":605,"num_write_kb":19082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":9405861,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"231'682","reported_seq":8096,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:25.704967+0000","last_change":"2024-09-16T11:17:25.704935+0000","last_active":"2024-09-16T11:17:25.704967+0000","last_peered":"2024-09-16T11:17:25.704967+0000","last_clean":"2024-09-16T11:17:25.704967+0000","last_became_active":"2024-09-16T11:16:39.491681+0000","last_became_peered":"2024-09-16T11:16:39.491681+0000","last_unstale":"2024-09-16T11:17:25.704967+0000","last_undegraded":"2024-09-16T11:17:25.704967+0000","last_fullsized":"2024-09-16T11:17:25.704967+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'682","last_scrub_stamp":"2024-09-16T11:17:25.704824+0000","last_deep_scrub":"231'682","last_deep_scrub_stamp":"2024-09-16T11:17:25.704824+0000","last_clean_scrub_stamp":"2024-09-16T11:17:25.704824+0000","objects_scrubbed":14,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:20:25.457976+0000","scrub_duration":0.021165478000000001,"objects_trimmed":2,"snaptrim_duration":0.006816356,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":8664,"num_read_kb":27395,"num_write":649,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":7510785,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"231'1216","reported_seq":10360,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:14.426230+0000","last_change":"2024-09-16T11:17:14.426073+0000","last_active":"2024-09-16T11:17:14.426230+0000","last_peered":"2024-09-16T11:17:14.426230+0000","last_clean":"2024-09-16T11:17:14.426230+0000","last_became_active":"2024-09-16T11:15:04.703440+0000","last_became_peered":"2024-09-16T11:15:04.703440+0000","last_unstale":"2024-09-16T11:17:14.426230+0000","last_undegraded":"2024-09-16T11:17:14.426230+0000","last_fullsized":"2024-09-16T11:17:14.426230+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1216","last_scrub_stamp":"2024-09-16T11:17:14.425792+0000","last_deep_scrub":"231'1216","last_deep_scrub_stamp":"2024-09-16T11:17:14.425792+0000","last_clean_scrub_stamp":"2024-09-16T11:17:14.425792+0000","objects_scrubbed":11,"log_size":1216,"log_dups_size":0,"ondisk_log_size":1216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:48.991766+0000","scrub_duration":0.020939989999999999,"objects_trimmed":0,"snaptrim_duration":0.0036247689999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12947,"num_read_kb":41174,"num_write":1878,"num_write_kb":19797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":31861757,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"231'1021","reported_seq":17317,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:35.247927+0000","last_change":"2024-09-16T11:17:35.247888+0000","last_active":"2024-09-16T11:17:35.247927+0000","last_peered":"2024-09-16T11:17:35.247927+0000","last_clean":"2024-09-16T11:17:35.247927+0000","last_became_active":"2024-09-16T11:15:09.475849+0000","last_became_peered":"2024-09-16T11:15:09.475849+0000","last_unstale":"2024-09-16T11:17:35.247927+0000","last_undegraded":"2024-09-16T11:17:35.247927+0000","last_fullsized":"2024-09-16T11:17:35.247927+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'1021","last_scrub_stamp":"2024-09-16T11:17:35.247780+0000","last_deep_scrub":"231'1021","last_deep_scrub_stamp":"2024-09-16T11:17:35.247780+0000","last_clean_scrub_stamp":"2024-09-16T11:17:35.247780+0000","objects_scrubbed":12,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:34:36.620529+0000","scrub_duration":0.023744536,"objects_trimmed":0,"snaptrim_duration":0.0065126669999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17085,"num_read_kb":67187,"num_write":1548,"num_write_kb":25063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":34124671,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"231'773","reported_seq":8830,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:17.327946+0000","last_change":"2024-09-16T11:17:17.327918+0000","last_active":"2024-09-16T11:17:17.327946+0000","last_peered":"2024-09-16T11:17:17.327946+0000","last_clean":"2024-09-16T11:17:17.327946+0000","last_became_active":"2024-09-16T11:15:07.454615+0000","last_became_peered":"2024-09-16T11:15:07.454615+0000","last_unstale":"2024-09-16T11:17:17.327946+0000","last_undegraded":"2024-09-16T11:17:17.327946+0000","last_fullsized":"2024-09-16T11:17:17.327946+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'773","last_scrub_stamp":"2024-09-16T11:17:17.327818+0000","last_deep_scrub":"231'773","last_deep_scrub_stamp":"2024-09-16T11:17:17.327818+0000","last_clean_scrub_stamp":"2024-09-16T11:17:17.327818+0000","objects_scrubbed":4,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:30.892162+0000","scrub_duration":0.012298748,"objects_trimmed":0,"snaptrim_duration":0.0054344110000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":9894,"num_read_kb":98011,"num_write":835,"num_write_kb":65983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":83511590,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"231'553","reported_seq":5287,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-16T11:17:18.458925+0000","last_change":"2024-09-16T11:17:18.458883+0000","last_active":"2024-09-16T11:17:18.458925+0000","last_peered":"2024-09-16T11:17:18.458925+0000","last_clean":"2024-09-16T11:17:18.458925+0000","last_became_active":"2024-09-16T11:16:47.485263+0000","last_became_peered":"2024-09-16T11:16:47.485263+0000","last_unstale":"2024-09-16T11:17:18.458925+0000","last_undegraded":"2024-09-16T11:17:18.458925+0000","last_fullsized":"2024-09-16T11:17:18.458925+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'553","last_scrub_stamp":"2024-09-16T11:17:18.458738+0000","last_deep_scrub":"231'553","last_deep_scrub_stamp":"2024-09-16T11:17:18.458738+0000","last_clean_scrub_stamp":"2024-09-16T11:17:18.458738+0000","objects_scrubbed":13,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:18.750700+0000","scrub_duration":0.019018033,"objects_trimmed":0,"snaptrim_duration":0.002945367,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5454,"num_read_kb":24267,"num_write":552,"num_write_kb":27395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":29934118,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":294,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":290,"num_read":253528,"num_read_kb":1427198,"num_write":28483,"num_write_kb":752588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13782,"num_bytes_recovered":829562294,"num_keys_recovered":1010,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0},"log_size":23767,"ondisk_log_size":23767,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":336,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":593,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":336,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":8,"acting":8,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":213,"seq":914828034064,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309620,"kb_used_data":3308,"kb_used_omap":37,"kb_used_meta":306266,"kb_avail":94062220,"statfs":{"total":96636764160,"available":96319713280,"internally_reserved":0,"allocated":3387392,"data_stored":2023048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38011,"internal_metadata":313617285},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":1.1619999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.2170000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":1.194},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.133}]},{"osd":3,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.878}]},{"osd":4,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.78500000000000003}]},{"osd":5,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.262},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.127}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304244,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94067596,"statfs":{"total":96636764160,"available":96325218304,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80900000000000005,"15min":0.83399999999999996},"min":{"1min":0.45800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.2310000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.90100000000000002,"15min":0.91300000000000003},"min":{"1min":0.53300000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.109}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0529999999999999,"5min":1.272,"15min":1.272},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85699999999999998,"15min":0.86899999999999999},"min":{"1min":0.56000000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3640000000000001,"5min":1.383,"15min":1.383},"last":1.181}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84499999999999997},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.109,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.93500000000000005,"15min":0.99399999999999999},"min":{"1min":0.61399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1899999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.86299999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90900000000000003,"15min":0.90700000000000003},"min":{"1min":0.622,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.27,"5min":2.302,"15min":2.302},"last":1.236},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.67800000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.298,"5min":1.466,"15min":1.466},"last":1.137}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.94199999999999995,"15min":0.99399999999999999},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2410000000000001,"5min":1.417,"15min":1.417},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.181,"15min":1.097},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.518,"5min":15.597,"15min":15.597},"last":1.288}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94599999999999995,"15min":0.91100000000000003},"min":{"1min":0.60199999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2569999999999999,"5min":1.603,"15min":1.603},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0840000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.343,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.337}]},{"osd":7,"last update":"Mon Sep 16 11:17:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":1.044},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300096,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071744,"statfs":{"total":96636764160,"available":96329465856,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83299999999999996,"15min":0.85299999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4810000000000001,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.294},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82799999999999996,"15min":0.86499999999999999},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.054,"5min":1.427,"15min":1.427},"last":0.86099999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.206,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.89700000000000002,"15min":0.90300000000000002},"min":{"1min":0.49299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.5900000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.996}]},{"osd":2,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93999999999999995,"15min":0.998},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3120000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.246},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.97899999999999998,"15min":1.0529999999999999},"min":{"1min":0.54000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0860000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.90100000000000002},"min":{"1min":0.495,"5min":0.437,"15min":0.437},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.93700000000000006,"15min":0.93300000000000005},"min":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2430000000000001,"5min":1.748,"15min":1.748},"last":1.1830000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.91600000000000004,"15min":0.93300000000000005},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.107,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.002,"15min":0.998},"min":{"1min":0.51800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.107}]},{"osd":6,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.94699999999999995,"15min":0.96399999999999997},"min":{"1min":0.55400000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.282,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.224,"15min":1.133},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4419999999999999,"5min":16.024000000000001,"15min":16.024000000000001},"last":1.343}]},{"osd":7,"last update":"Mon Sep 16 11:17:44 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.137},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.97099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312576,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94059264,"statfs":{"total":96636764160,"available":96316686336,"internally_reserved":0,"allocated":2809856,"data_stored":1433411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.73399999999999999,"15min":0.755},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0920000000000001,"5min":1.329,"15min":1.329},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.81699999999999995,"15min":0.81299999999999994},"min":{"1min":0.41899999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.0660000000000001}]},{"osd":1,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84899999999999998,"15min":0.89500000000000002},"min":{"1min":0.48799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.751,"15min":0.78000000000000003},"min":{"1min":0.54700000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.79600000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84599999999999997,"15min":0.90900000000000003},"min":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1679999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.139},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.98199999999999998,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.278,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.2270000000000001}]},{"osd":3,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.85199999999999998,"15min":0.872},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1990000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":1.107},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.97999999999999998,"15min":1.036},"min":{"1min":0.66200000000000003,"5min":0.432,"15min":0.432},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.1930000000000001}]},{"osd":5,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.99099999999999999,"15min":1.0649999999999999},"min":{"1min":0.499,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.4379999999999999,"5min":2.3879999999999999,"15min":2.3879999999999999},"last":1.159},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.94499999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.25,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.4330000000000001}]},{"osd":6,"last update":"Mon Sep 16 11:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.028,"15min":1.1160000000000001},"min":{"1min":0.51100000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3740000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.101,"15min":1.1359999999999999},"min":{"1min":0.66100000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.508,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":1.403}]},{"osd":7,"last update":"Mon Sep 16 11:17:43 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":1.26},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.3029999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300096,"kb_used_data":2744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94071744,"statfs":{"total":96636764160,"available":96329465856,"internally_reserved":0,"allocated":2809856,"data_stored":1433415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.873,"5min":1.478,"15min":1.478},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.71999999999999997,"15min":0.66600000000000004},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.80400000000000005}]},{"osd":1,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.032,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.44800000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":4.25,"5min":4.25,"15min":4.25},"last":0.92400000000000004}]},{"osd":2,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1739999999999999,"5min":2.105,"15min":2.105},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.099,"5min":0.877,"15min":0.82499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.99399999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.89500000000000002},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.107,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.97799999999999998,"15min":0.99399999999999999},"min":{"1min":0.63900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4159999999999999,"5min":2.153,"15min":2.153},"last":0.84399999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.75900000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":3.5990000000000002,"5min":3.5990000000000002,"15min":3.5990000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.024},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.95399999999999996}]},{"osd":6,"last update":"Mon Sep 16 11:16:55 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.052,"15min":1.079},"min":{"1min":0.65100000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4910000000000001,"5min":2.4590000000000001,"15min":2.4590000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.097,"15min":1.0609999999999999},"min":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.2949999999999999,"5min":4.2949999999999999,"15min":4.2949999999999999},"last":1.1220000000000001}]},{"osd":7,"last update":"Mon Sep 16 11:17:40 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.0960000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308404,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063436,"statfs":{"total":96636764160,"available":96320958464,"internally_reserved":0,"allocated":2797568,"data_stored":1433226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70199999999999996,"15min":0.69199999999999995},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.083,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.875}]},{"osd":1,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61399999999999999,"15min":0.58599999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71899999999999997,"15min":0.76100000000000001},"min":{"1min":0.40899999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.3460000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.0069999999999999}]},{"osd":3,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.746,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.147,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.40500000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.113,"5min":1.361,"15min":1.361},"last":0.75800000000000001}]},{"osd":4,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90900000000000003,"15min":0.94299999999999995},"min":{"1min":0.5,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.466,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89400000000000002,"15min":0.89900000000000002},"min":{"1min":0.496,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.96399999999999997}]},{"osd":5,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.63800000000000001,"5min":0.498,"15min":0.498},"max":{"1min":5.21,"5min":5.21,"15min":5.21},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.181,"5min":1.0600000000000001,"15min":1.0649999999999999},"min":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":5.25,"5min":5.25,"15min":5.25},"last":1.25}]},{"osd":6,"last update":"Mon Sep 16 11:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.004,"15min":1.056},"min":{"1min":0.53700000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.954,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.038},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4930000000000001,"5min":1.744,"15min":1.744},"last":1.0649999999999999}]},{"osd":7,"last update":"Mon Sep 16 11:17:41 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.2210000000000001,"5min":2.2210000000000001,"15min":2.2210000000000001},"last":1.196}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308408,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063432,"statfs":{"total":96636764160,"available":96320954368,"internally_reserved":0,"allocated":2801664,"data_stored":1433245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73099999999999998,"15min":0.751},"min":{"1min":0.42499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.78800000000000003,"15min":0.83999999999999997},"min":{"1min":0.499,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.78400000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.47299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.3129999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.319,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.83599999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.77500000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0660000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.79700000000000004,"15min":0.79200000000000004},"min":{"1min":0.55300000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.484,"5min":1.784,"15min":1.784},"last":1.0089999999999999}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.90600000000000003,"15min":0.94999999999999996},"min":{"1min":0.64900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.036},{"interface":"front","average":{"1min":1.095,"5min":1.0149999999999999,"15min":1.04},"min":{"1min":0.67800000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.611,"5min":2.597,"15min":2.597},"last":1.095}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0209999999999999,"15min":1.091},"min":{"1min":0.55400000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.6399999999999999,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.98599999999999999,"15min":1.024},"min":{"1min":0.61699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.347,"5min":2.266,"15min":2.266},"last":0.749}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0549999999999999,"15min":1.121},"min":{"1min":0.65100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2769999999999999,"5min":2.552,"15min":2.552},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.042,"15min":1.1040000000000001},"min":{"1min":0.52500000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.79,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:17:41 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.171}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313152,"kb_used_data":3320,"kb_used_omap":1,"kb_used_meta":309822,"kb_avail":94058688,"statfs":{"total":96636764160,"available":96316096512,"internally_reserved":0,"allocated":3399680,"data_stored":2023760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317258072},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.40000000000000002,"5min":0.215,"15min":0.215},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67900000000000005,"15min":0.65500000000000003},"min":{"1min":0.501,"5min":0.318,"15min":0.318},"max":{"1min":0.91700000000000004,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.85199999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68200000000000005,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.70599999999999996,"15min":0.67800000000000005},"min":{"1min":0.46000000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0780000000000001,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.41199999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.24,"5min":1.946,"15min":1.946},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.89600000000000002,"15min":0.96199999999999997},"min":{"1min":0.51300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":3.2309999999999999,"15min":3.2309999999999999},"last":0.80300000000000005}]},{"osd":4,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.874,"15min":0.86199999999999999},"min":{"1min":0.56699999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.27,"5min":1.365,"15min":1.365},"last":1.056},{"interface":"front","average":{"1min":1.006,"5min":0.95099999999999996,"15min":0.97499999999999998},"min":{"1min":0.70899999999999996,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.329,"5min":2.129,"15min":2.129},"last":0.89700000000000002}]},{"osd":5,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94399999999999995,"15min":1.006},"min":{"1min":0.63700000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.486,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.04,"15min":1.103},"min":{"1min":0.78400000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.123}]},{"osd":6,"last update":"Mon Sep 16 11:16:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.022,"15min":1.024},"min":{"1min":0.79400000000000004,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.016},{"interface":"front","average":{"1min":1.091,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.76400000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.538,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.94599999999999995}]},{"osd":7,"last update":"Mon Sep 16 11:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.159},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.216}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9985,"internal_metadata":0}]}} 2024-09-16T11:17:49.663 DEBUG:teuthology.orchestra.run.smithi162:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T11:17:49.977 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T11:17:49.977 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T11:17:49.977 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T11:17:49.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.059 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T11:17:50.059 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T11:17:50.059 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.156 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T11:17:50.157 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T11:17:50.157 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.236 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T11:17:50.236 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T11:17:50.236 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.303 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T11:17:50.303 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T11:17:50.303 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.389 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T11:17:50.389 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T11:17:50.390 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.474 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T11:17:50.474 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T11:17:50.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.551 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T11:17:50.551 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T11:17:50.551 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.606 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T11:17:50.607 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T11:17:50.607 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T11:17:50.607 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.633 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T11:17:50.634 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T11:17:50.634 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.659 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T11:17:50.660 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T11:17:50.660 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T11:17:50.660 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.680 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T11:17:50.681 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T11:17:50.681 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.742 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T11:17:50.742 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T11:17:50.742 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:17:50.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-16T11:17:50.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-16T11:17:50.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-16T11:17:50.761 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T11:17:50.761 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T11:17:50.762 DEBUG:teuthology.orchestra.run.smithi162:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-16T11:17:50.785 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:17:50.786 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T11:17:51.116 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:17:51.116 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T11:17:51.416 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:17:51.416 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T11:17:51.622 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:17:51.622 DEBUG:teuthology.orchestra.run.smithi162:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T11:17:51.859 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:17:51.859 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T11:17:52.116 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:17:52.279 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T11:17:52.471 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:17:52.471 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T11:17:52.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:17:52.652 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T11:17:52.820 INFO:tasks.ceph:Archiving mon data... 2024-09-16T11:17:52.821 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/data/mon.a.tgz 2024-09-16T11:17:52.822 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T11:17:52.828 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.PmXsPU3Bxv 2024-09-16T11:17:52.829 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PmXsPU3Bxv 2024-09-16T11:17:52.953 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.PmXsPU3Bxv 2024-09-16T11:17:53.021 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.PmXsPU3Bxv is 472KB 2024-09-16T11:17:53.046 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.PmXsPU3Bxv 2024-09-16T11:17:53.052 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/data/mon.c.tgz 2024-09-16T11:17:53.053 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-16T11:17:53.099 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.0VJvrtae8c 2024-09-16T11:17:53.100 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0VJvrtae8c 2024-09-16T11:17:53.262 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.0VJvrtae8c 2024-09-16T11:17:53.285 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.0VJvrtae8c is 662KB 2024-09-16T11:17:53.348 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.0VJvrtae8c 2024-09-16T11:17:53.355 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/data/mon.b.tgz 2024-09-16T11:17:53.356 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-09-16T11:17:53.362 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.JE2imofH6A 2024-09-16T11:17:53.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.JE2imofH6A 2024-09-16T11:17:53.570 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.JE2imofH6A 2024-09-16T11:17:53.597 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.JE2imofH6A is 662KB 2024-09-16T11:17:53.661 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.JE2imofH6A 2024-09-16T11:17:53.668 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T11:17:53.669 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T11:17:53.671 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T11:17:53.686 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T11:17:53.686 DEBUG:teuthology.orchestra.run.smithi162:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T11:17:53.731 DEBUG:teuthology.orchestra.run.smithi165:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T11:17:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T11:17:53.763 INFO:teuthology.orchestra.run.smithi162.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T11:17:53.770 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T11:17:53.772 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/remote/smithi162/crash 2024-09-16T11:17:53.772 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T11:17:53.787 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/remote/smithi165/crash 2024-09-16T11:17:53.788 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T11:17:53.818 INFO:tasks.ceph:Compressing logs... 2024-09-16T11:17:53.818 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T11:17:53.821 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T11:17:53.851 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27919.log 2024-09-16T11:17:53.851 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-09-16T11:17:53.852 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.27919.log: -- /var/log/ceph/ceph-client.admin.41786.log 2024-09-16T11:17:53.852 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27919.log.gz 2024-09-16T11:17:53.852 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32025.log 2024-09-16T11:17:53.854 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41354.log: 0.0%/var/log/ceph/ceph-client.admin.41786.log: -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-09-16T11:17:53.854 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28397.log 2024-09-16T11:17:53.854 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41786.log.gz 2024-09-16T11:17:53.854 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32824.log 2024-09-16T11:17:53.854 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32025.log.gz 2024-09-16T11:17:53.854 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40536.log 2024-09-16T11:17:53.855 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28397.log: /var/log/ceph/ceph-client.admin.32824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28397.log.gz 2024-09-16T11:17:53.855 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32824.log.gz 2024-09-16T11:17:53.855 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40863.log 2024-09-16T11:17:53.855 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37436.log 2024-09-16T11:17:53.857 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40536.log.gz 2024-09-16T11:17:53.857 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28995.log 2024-09-16T11:17:53.857 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40863.log.gz 2024-09-16T11:17:53.857 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29081.log 2024-09-16T11:17:53.860 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37436.log.gz 2024-09-16T11:17:53.860 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33058.log 2024-09-16T11:17:53.860 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28995.log.gz 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22237.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22146.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22237.log.gz 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.5.log: gzip /var/log/ceph/ceph-client.admin.22146.log: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22146.log.gz 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22314.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T11:17:53.866 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31102.log 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22196.log /var/log/ceph/ceph-osd.6.log: 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22314.log.gz 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:gzip /var/log/ceph/objectstore_tool.31102.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.22264.log 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22196.log.gz 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22339.log 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22264.log.gz 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T11:17:53.867 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.22339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22339.log.gz 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22289.log 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22171.log 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.22289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22289.log.gz 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr:gzip /var/log/ceph/ceph-client.admin.22171.log: -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22171.log.gz 83.9% 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T11:17:53.874 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-osd.7.log: 95.5% -- replaced with /var/log/ceph/objectstore_tool.31102.log.gz 2024-09-16T11:17:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30763.log 2024-09-16T11:17:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29081.log.gz 2024-09-16T11:17:53.906 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28863.log 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33058.log.gz 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41840.log 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30763.log.gz 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28428.log 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28863.log.gz 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40455.log 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41840.log.gz 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28428.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28428.log.gz 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35659.log 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40455.log.gz 2024-09-16T11:17:53.907 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39305.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28915.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35659.log.gz 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31304.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39305.log.gz 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.28915.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41678.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28915.log.gz 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33450.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31304.log.gz 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40890.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41678.log.gz 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41111.log 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33450.log.gz 2024-09-16T11:17:53.908 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40890.log.gz 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.41111.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41111.log.gz 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31240.log 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38242.log 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34999.log: /var/log/ceph/ceph-client.admin.31240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31240.log.gz 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36059.log 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39605.log 2024-09-16T11:17:53.909 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36059.log: /var/log/ceph/ceph-client.admin.38242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36334.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38242.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36059.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41516.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39605.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35381.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36334.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28176.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41516.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30433.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35381.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40347.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28176.log.gz 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40099.log 2024-09-16T11:17:53.910 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30433.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40347.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35408.log 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40099.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36417.log 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34659.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31180.log 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35408.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36417.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31180.log: /var/log/ceph/ceph-client.admin.40671.log 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31180.log.gz 2024-09-16T11:17:53.911 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31040.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36926.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40671.log: /var/log/ceph/ceph-client.admin.31040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31040.log.gz 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39035.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36614.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36926.log.gz 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27957.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39035.log.gz 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35070.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36614.log: /var/log/ceph/ceph-client.admin.27957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36614.log.gz 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27957.log.gz-5 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:--verbose -- /var/log/ceph/ceph-client.admin.31097.log 2024-09-16T11:17:53.912 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38233.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35070.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38075.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31097.log: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.31097.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27892.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38233.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30382.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38075.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34974.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27892.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22385.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30382.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41083.log 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34974.log.gz 2024-09-16T11:17:53.913 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.22385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41570.log 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22385.log.gz 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32705.log 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41083.log.gz 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41651.log 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41570.log.gz 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28729.log 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32705.log.gz 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39084.log 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41651.log.gz 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40874.log 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28729.log.gz 2024-09-16T11:17:53.914 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32975.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39084.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29663.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40874.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32975.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.29663.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28386.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29663.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37409.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29128.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28386.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32637.log 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37409.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-09-16T11:17:53.915 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37280.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29128.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28191.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32637.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33931.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37280.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29620.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28191.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29806.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33931.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35631.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29620.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.29806.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40482.log 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29806.log.gz 2024-09-16T11:17:53.916 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39510.log 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35631.log.gz 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34295.log 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40482.log.gz 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31148.log 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39510.log.gz 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28594.log 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34295.log.gz 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36651.log 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31148.log.gz 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28594.log.gz 2024-09-16T11:17:53.917 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36651.log.gz 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35965.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29876.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30055.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35965.log.gz 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29876.log.gz 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30055.log.gz 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-09-16T11:17:53.918 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.36166.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27965.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28192.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30236.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27965.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28702.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28192.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.30236.log: -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30236.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 /var/log/ceph/ceph-client.admin.28702.log: --verbose -- /var/log/ceph/ceph-client.admin.36797.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28702.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28837.log 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-16T11:17:53.919 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35505.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36797.log.gz 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33487.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28837.log.gz 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37919.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35505.log.gz 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35435.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33487.log.gz 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29501.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37919.log.gz 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32217.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35435.log.gz 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27972.log 2024-09-16T11:17:53.920 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29501.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28598.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32217.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30768.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27972.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34253.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28598.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28809.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30768.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29049.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34253.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34936.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28809.log.gz 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37564.log 2024-09-16T11:17:53.921 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29049.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39931.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34936.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34712.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37564.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40672.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39931.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30639.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34712.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34095.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40672.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36124.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30639.log.gz 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30915.log 2024-09-16T11:17:53.922 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41435.log 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34095.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36124.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.30915.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29124.log 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30915.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32376.log 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41435.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28941.log 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29124.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27865.log 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32376.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33150.log 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28941.log.gz 2024-09-16T11:17:53.923 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27865.log.gz 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40018.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33150.log.gzgzip 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.30340.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40018.log.gz 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38784.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gz 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30340.log.gz 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38784.log.gz 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-09-16T11:17:53.924 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31761.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40320.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31761.log.gz 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32345.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40320.log.gz 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41813.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38678.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32345.log.gz 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28406.log 2024-09-16T11:17:53.925 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41813.log.gz 2024-09-16T11:17:53.926 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-09-16T11:17:53.926 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31894.log 2024-09-16T11:17:53.926 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38678.log.gz 2024-09-16T11:17:53.926 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28406.log.gz 2024-09-16T11:17:53.926 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-09-16T11:17:53.963 INFO:teuthology.orchestra.run.smithi162.stderr: 93.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T11:17:53.963 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37537.log 2024-09-16T11:17:53.963 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.31894.log: -- /var/log/ceph/ceph-client.admin.28190.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31894.log.gz 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29845.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37537.log.gz 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38596.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28190.log.gz 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28400.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29845.log.gz 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41327.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31958.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38596.log.gz 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28400.log: gzip /var/log/ceph/ceph-client.admin.41327.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29422.log 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28400.log.gz 2024-09-16T11:17:53.964 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41327.log.gz 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32735.log 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31958.log.gz 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41297.log 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29422.log.gz 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40401.log 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32735.log.gz 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40877.log 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41297.log.gz 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28889.log 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40401.log.gz 2024-09-16T11:17:53.965 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41543.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40877.log.gz 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28382.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28889.log.gz 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33359.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41543.log.gz 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35798.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28382.log.gz 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22082.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33359.log.gz 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33032.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29872.log 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.22082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35798.log.gz 2024-09-16T11:17:53.966 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22082.log.gz 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40428.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33032.log.gz 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29872.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.29872.log.gz--verbose -- 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27819.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27971.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0%/var/log/ceph/ceph-client.admin.40880.log: -- replaced with /var/log/ceph/ceph-client.admin.40428.log.gz 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 /var/log/ceph/ceph-client.admin.27819.log: --verbose -- /var/log/ceph/ceph-client.admin.41759.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27819.log.gz 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38483.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27971.log.gz 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36695.log 2024-09-16T11:17:53.967 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41759.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37854.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38483.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36695.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37854.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28603.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31461.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40590.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28603.log.gz 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27793.log 2024-09-16T11:17:53.968 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31461.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37253.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40590.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34021.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27793.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37253.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30538.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34021.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32088.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28169.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30538.log.gz 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37944.log 2024-09-16T11:17:53.969 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32088.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38982.log 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28169.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31632.log 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37944.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41408.log 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38982.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33564.log 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31632.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39636.log 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41408.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29464.log 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33564.log: 28.9% -- replaced with /var/log/ceph/ceph-client.admin.33564.log.gz 2024-09-16T11:17:53.970 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41867.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39636.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29341.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29464.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42027.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41867.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27974.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29341.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30184.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.42027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42027.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27974.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31434.log 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30184.log.gz 2024-09-16T11:17:53.971 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40509.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32495.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31434.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41975.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40509.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.32495.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41103.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32495.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31579.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41975.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40239.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41103.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.31579.log: -- /var/log/ceph/ceph-osd.1.log 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31579.log.gz 2024-09-16T11:17:53.972 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40668.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40239.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40239.log.gz 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30977.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.40668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41624.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40668.log.gz 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34530.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30977.log.gz 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41624.log: /var/log/ceph/ceph-client.admin.28164.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41624.log.gz 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40185.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34530.log.gz 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40126.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28164.log.gz 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34401.log 2024-09-16T11:17:53.973 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40185.log.gz 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39803.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40126.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.40126.log.gz--verbose 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:-- /var/log/ceph/ceph-client.admin.32429.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34401.log.gz 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39803.log.gz 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36361.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32429.log.gz 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35029.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36361.log.gz 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.35029.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36290.log 2024-09-16T11:17:53.974 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35029.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41921.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41381.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36290.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:gzip /var/log/ceph/ceph-client.admin.41921.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27967.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41921.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41381.log: /var/log/ceph/ceph-client.admin.37892.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41381.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31503.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27967.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32912.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37892.log.gz 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33760.log 2024-09-16T11:17:53.975 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31503.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28968.log 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32912.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33760.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31787.log 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28968.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38535.log 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36951.log 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31787.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28379.log 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38535.log.gz 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39262.log /var/log/ceph/ceph-client.admin.36951.log: 2024-09-16T11:17:53.976 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36951.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36452.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28379.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40866.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39262.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29290.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36452.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28675.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40866.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34127.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29290.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33940.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28675.log.gz 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29992.log 2024-09-16T11:17:53.977 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34127.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38709.log 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33940.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29992.log: /var/log/ceph/ceph-client.admin.41106.log 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29992.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38709.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33841.log 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41106.log: /var/log/ceph/ceph-client.admin.37200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41106.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33815.log 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29125.log 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33841.log.gz 2024-09-16T11:17:53.978 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34602.log 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33815.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40617.log 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29125.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32465.log 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34602.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.35074.log -- replaced with /var/log/ceph/ceph-client.admin.34602.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40617.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40158.log 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32465.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35074.log: /var/log/ceph/ceph-client.admin.40158.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35074.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40158.log.gz 2024-09-16T11:17:53.979 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38393.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz/var/log/ceph/ceph-client.admin.38393.log: 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29115.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38393.log.gz 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.42002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30095.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34030.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.0.29115.log: /var/log/ceph/ceph-client.admin.30095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38327.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30095.log.gz 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34030.log.gz 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.29115.log.gz 2024-09-16T11:17:53.980 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29774.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38327.log: /var/log/ceph/ceph-client.admin.40670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38327.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39758.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38034.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29774.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36170.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39758.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28783.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38034.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34332.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36170.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35280.log 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28783.log.gz 2024-09-16T11:17:53.981 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29749.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34332.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32248.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35280.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29749.log: /var/log/ceph/ceph.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29749.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40663.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40870.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32248.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41489.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40663.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40870.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37082.log 2024-09-16T11:17:53.982 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30666.log 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41489.log.gz 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37082.log.gz 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34488.log 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30666.log.gz 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29552.log 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34488.log.gz 2024-09-16T11:17:53.983 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29552.log.gz 2024-09-16T11:17:54.008 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31683.log 2024-09-16T11:17:54.008 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28170.log 2024-09-16T11:17:54.008 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31683.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41948.log 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31683.log.gz 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39871.log 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28170.log.gz 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41948.log.gz 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35844.log 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39871.log.gz 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33673.log 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38083.log 2024-09-16T11:17:54.009 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35844.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35190.log 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33673.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27966.log 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38083.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36749.log 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38083.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35190.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27966.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.39188.log -- replaced with /var/log/ceph/ceph-client.admin.27966.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31310.log 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36749.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41732.log 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39188.log.gz 2024-09-16T11:17:54.010 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34821.log 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31310.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41732.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30856.log 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34821.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37772.log 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30856.log: -- replaced with /var/log/ceph/ceph-client.admin.34821.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30856.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36225.log 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30211.log 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37772.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37772.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36225.log.gz 2024-09-16T11:17:54.011 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35559.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41086.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30211.log.gz 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29022.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35559.log.gz 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41086.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17273.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41086.log.gz 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33258.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29022.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39376.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph.tmp-client.admin.17273.log: 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17273.log.gz 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32130.log 2024-09-16T11:17:54.012 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33258.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.39376.log: -- /var/log/ceph/ceph-client.admin.35914.log 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39376.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40563.log 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32130.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34186.log 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35914.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37683.log 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40563.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34186.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36055.log 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37683.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37683.log.gz 2024-09-16T11:17:54.013 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38836.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36055.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32876.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.38836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38836.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28178.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36842.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32876.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34358.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28178.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33874.log 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36842.log: 26.9% -- replaced with /var/log/ceph/ceph-client.admin.36842.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.34358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34358.log.gz 2024-09-16T11:17:54.014 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37607.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33874.log.gz 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40045.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22246.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37607.log.gz 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40072.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27845.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40045.log.gz 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.22246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.22246.log.gz 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40072.log.gz 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27845.log.gz 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27719.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40669.log 2024-09-16T11:17:54.015 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39707.log 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.27719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27719.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30511.log 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40669.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39707.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28395.log 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.30511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30511.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39472.log 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29929.log 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28395.log.gz 2024-09-16T11:17:54.016 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41894.log 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39472.log.gz 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37319.log 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.29929.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33730.log 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29929.log.gz 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41894.log.gz 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37319.log: /var/log/ceph/ceph-client.admin.33584.log 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37319.log.gz 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31893.log 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37736.log 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33730.log.gz/var/log/ceph/ceph-client.admin.33584.log: 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33584.log.gz 2024-09-16T11:17:54.017 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.31893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35871.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36388.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37736.log.gz 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35871.log.gz 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27973.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.36388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36388.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.41597.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40667.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz/var/log/ceph/ceph-client.admin.27973.log: 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-09-16T11:17:54.018 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-osd.2.log: -- replaced with /var/log/ceph/ceph-client.admin.27973.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.40667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40667.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33328.log 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39379.log 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.33328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32576.log 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33328.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.39379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39379.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37997.log 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.32576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32576.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28755.log 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.37997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35441.log 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37997.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.28755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28755.log.gz 2024-09-16T11:17:54.019 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/ceph-client.admin.35441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35441.log.gz 2024-09-16T11:17:54.054 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T11:17:54.098 INFO:teuthology.orchestra.run.smithi162.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T11:17:54.325 INFO:teuthology.orchestra.run.smithi162.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T11:17:54.628 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T11:17:54.907 INFO:teuthology.orchestra.run.smithi165.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T11:17:55.753 INFO:teuthology.orchestra.run.smithi162.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T11:17:55.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-16T11:17:55.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-16T11:17:55.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-16T11:17:55.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-16T11:17:55.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-16T11:17:55.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-16T11:17:55.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-16T11:17:55.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-16T11:17:55.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-16T11:17:55.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-16T11:17:56.894 INFO:teuthology.orchestra.run.smithi165.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T11:17:57.642 INFO:teuthology.orchestra.run.smithi162.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T11:17:59.254 INFO:teuthology.orchestra.run.smithi165.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T11:17:59.255 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-16T11:17:59.255 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m5.429s 2024-09-16T11:17:59.255 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m10.099s 2024-09-16T11:17:59.255 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m0.397s 2024-09-16T11:17:59.360 INFO:teuthology.orchestra.run.smithi162.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T11:17:59.509 INFO:teuthology.orchestra.run.smithi162.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T11:17:59.510 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-16T11:17:59.510 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m5.686s 2024-09-16T11:17:59.510 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m16.783s 2024-09-16T11:17:59.510 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m0.725s 2024-09-16T11:17:59.511 INFO:tasks.ceph:Archiving logs... 2024-09-16T11:17:59.512 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/remote/smithi162/log 2024-09-16T11:17:59.513 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T11:18:01.948 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/remote/smithi165/log 2024-09-16T11:18:01.949 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T11:18:03.323 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T11:18:03.333 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T11:18:03.333 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T11:18:03.336 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T11:18:03.405 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T11:18:03.415 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T11:18:03.416 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:18:03.419 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:18:03.441 INFO:teuthology.orchestra.run.smithi162.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T11:18:03.441 INFO:teuthology.orchestra.run.smithi162.stdout:============================================================================== 2024-09-16T11:18:03.441 INFO:teuthology.orchestra.run.smithi162.stdout:*hv01.front.sepi 67.205.162.81 3 u 32 64 377 0.086 0.085 0.689 2024-09-16T11:18:03.442 INFO:teuthology.orchestra.run.smithi162.stdout:+hv02.front.sepi 63.231.80.2 3 u 37 64 377 0.080 -2.036 0.820 2024-09-16T11:18:03.442 INFO:teuthology.orchestra.run.smithi162.stdout:+hv03.front.sepi 74.6.168.72 3 u 30 64 377 0.086 0.302 0.725 2024-09-16T11:18:03.442 INFO:teuthology.orchestra.run.smithi162.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:18:03.444 INFO:teuthology.orchestra.run.smithi165.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T11:18:03.444 INFO:teuthology.orchestra.run.smithi165.stdout:============================================================================== 2024-09-16T11:18:03.444 INFO:teuthology.orchestra.run.smithi165.stdout:*hv01.front.sepi 67.205.162.81 3 u 37 64 377 0.064 -2.992 4.683 2024-09-16T11:18:03.444 INFO:teuthology.orchestra.run.smithi165.stdout:+hv02.front.sepi 63.231.80.2 3 u 36 64 377 0.057 1.511 3.463 2024-09-16T11:18:03.444 INFO:teuthology.orchestra.run.smithi165.stdout:+hv03.front.sepi 74.6.168.72 3 u 38 64 377 0.093 1.200 2.056 2024-09-16T11:18:03.444 INFO:teuthology.orchestra.run.smithi165.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-16T11:18:03.445 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T11:18:03.456 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T11:18:03.457 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T11:18:03.466 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T11:18:03.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T11:18:03.483 INFO:teuthology.task.internal:Duration was 846.742214 seconds 2024-09-16T11:18:03.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T11:18:03.492 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T11:18:03.492 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T11:18:03.495 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T11:18:03.547 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T11:18:03.547 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-09-16T11:18:03.547 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T11:18:03.561 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-09-16T11:18:03.561 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T11:18:03.576 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T11:18:03.576 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T11:18:03.579 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T11:18:03.618 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T11:18:03.618 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T11:18:03.663 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T11:18:03.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T11:18:03.772 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T11:18:03.772 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T11:18:03.787 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T11:18:03.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T11:18:03.815 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:18:03.831 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:18:03.843 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-09-16T11:18:03.861 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-09-16T11:18:03.889 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:18:03.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:18:03.895 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:18:03.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:18:03.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T11:18:03.911 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T11:18:03.912 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/remote/smithi162 2024-09-16T11:18:03.912 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T11:18:03.944 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907154/remote/smithi165 2024-09-16T11:18:03.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T11:18:03.968 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T11:18:03.969 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T11:18:03.983 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T11:18:04.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T11:18:04.026 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T11:18:04.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T11:18:04.044 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T11:18:04.044 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T11:18:04.046 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T11:18:04.052 INFO:teuthology.orchestra.run.smithi162.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 11:18 /home/ubuntu/cephtest 2024-09-16T11:18:04.054 INFO:teuthology.orchestra.run.smithi165.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 11:18 /home/ubuntu/cephtest 2024-09-16T11:18:04.056 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T11:18:04.066 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T11:18:04.101 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_fsx_copy_on_read} duration: 846.7422144412994 flavor: default owner: scheduled_sangadi@teuthology success: true 2024-09-16T11:18:04.104 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T11:18:04.182 INFO:teuthology.run:pass